The LangChain community has made significant strides in improving tooling and model quality for building with Large Language Models (LLMs). However, the abundance of options makes it challenging to separate signal from noise, emphasizing the need for reliable and relevant benchmarks. To address this, LangSmith is launching a platform that enables community-driven evaluation and benchmarking of LLM architectures on various tasks. The new `langchain-benchmarks` package facilitates experimentation and benchmarking for key functionality when building with LLMs. By sharing evaluation datasets and results, the community can collaborate to establish standards for evaluating LLM performance. This initiative aims to make it easier for developers to build with LLMs by providing a standardized framework for comparing approaches and identifying tradeoffs.