Evaluations (evals) are crucial for ensuring the reliability and quality of applications utilizing large language models (LLMs), and integrating testing frameworks like Pytest and Vitest/Jest with LangSmith offers a familiar environment to conduct these assessments. The LangSmith integrations, available in beta with v0.3.0 of the LangSmith SDKs, enable developers to leverage Pytest/Vitest's runtime behavior and LangSmith's observability and sharing features, allowing for nuanced metric logging beyond simple pass/fail outcomes. This setup aids in debugging the non-deterministic nature of LLMs, tracking progress over time, and fostering team collaboration by sharing experiment results. Python users benefit from built-in evaluation functions, such as expect.edit_distance(), to assess LLM output accuracy. Additionally, the integrations offer specific evaluation logic per test case, real-time feedback during local development, and CI pipeline compatibility to preempt regressions. LangSmith encourages developers to explore their tutorials and guides to implement this new approach to running evals and participate in the LangChain Slack Community for further engagement.