Company
Date Published
Author
Doug Safreno
Word count
821
Language
English
Hacker News points
None

Summary

Gentrace is a company focused on enhancing the development of large language model (LLM) products by creating robust evaluation systems, known as evals, which are crucial but often unreliable. The company has collaborated with notable clients, such as Quizlet and Webflow, and received backing from prominent angel investors. Gentrace's newly launched Experiments platform provides a collaborative testing environment, allowing LLM product and engineering teams to conduct systematic testing and fine-tune applications in real-time by adjusting various parameters. This platform integrates seamlessly with complex enterprise environments, helping teams build reliable test suites and involve multiple stakeholders, including domain experts, in the evaluation process. The company's future plans include expanding their evaluation capabilities with features like threshold-based experiments and prompt auto-optimization, aiming to further support the development of high-quality LLM products.