LangKit: Making Large Language Models Safe and Responsible

LangKit is a solution developed by WhyLabs for understanding generative models like Large Language Models (LLMs). It enables users to monitor the behavior and performance of their LLMs, ensuring their reliability, safety, and effectiveness. With LangKit, AI practitioners can extract critical telemetry data from prompts and responses, which can be used to help direct the behavior of an LLM through better prompt engineering and systematically observe at scale. The tool allows users to establish thresholds and baselines for a range of activities such as malicious prompts, sensitive data leakage, toxicity, problematic topics, hallucinations, and jailbreak attempts. LangKit is simple and extensible, with the ability to extract all important telemetry about an LLM with just a prompt and a response. It also supports User Defined Functions (UDFs) for users to add their own metrics or validate prompts and responses in a particular way.