June 2026 Summaries

Over the past few months, the author has attended several conferences, including PyData London, PyCon DE & PyData in Darmstadt, and PyCon Italia in Bologna, noting a significant shift in the focus of these events. A detailed analysis of talk abstracts revealed a sharp increase in sessions about agents and agentic patterns, while beginner and education content has notably decreased, indicating a shift in community interest and possibly reflecting broader industry trends. The rise in the importance of observability and evaluation is highlighted, as these tools are becoming crucial in managing the complexities of AI systems. The author expresses concerns about the current practices in agentic engineering, suggesting that the push for more iterations and complexity in AI processes may not necessarily lead to better outcomes, drawing parallels to outdated waterfall models of planning. There's a discussion on the challenges of balancing determinism and inference in AI systems, emphasizing the need for a mental model shift rather than just a tooling change among engineers. Furthermore, the topic of digital sovereignty is gaining traction, especially in European contexts, as regulatory frameworks like the EU AI Act start to impact AI development. The author notes a growing sense of anxiety and fatigue among engineers as they navigate these evolving landscapes, and stresses the importance of community and open conversations in addressing these challenges.

Pydantic Logfire introduced a self-hosted option over a year ago to accommodate customers who cannot use cloud environments due to regulatory or architectural constraints. This decision required adapting the software to run on customer-managed Kubernetes clusters using Helm, a familiar tool for Kubernetes teams. The transition highlighted challenges, such as increased customer support due to configuration mismatches during upgrades and resource allocation issues leading to performance problems. To address these, Logfire implemented an audit tool to synchronize platform configurations with Helm charts and introduced sizing presets to optimize resource allocation. Additionally, the need for explicit boundaries and APIs for instance management became apparent, as customers now manage their own deployments. The experience underscored the importance of treating deployment configurations as integral components of the product to reduce support overhead.

Harnesses are critical components that enhance the functionality and reliability of AI agents, transforming their outputs from mere predictions into verified and durable results. By surrounding the agent loop, a harness enables models to perform long-running tasks effectively by using tools safely, maintaining memory across calls, and managing context to ensure relevant information is accessible at the right moment. The key to a successful harness lies in its timing, allowing instructions and tools to be deferred until they are needed, thereby optimizing resource use and avoiding context overload. A harness also incorporates steering mechanisms to detect and correct errors promptly, minimizing the distance between a mistake and its resolution. In the context of Pydantic AI v2, capabilities are used to integrate instructions, tools, and settings into composable units that adapt dynamically during a run, allowing for a flexible and efficient workflow. The evolution towards a loop of agents, as opposed to a single harnessed agent, involves a system that autonomously assembles agents, adapts to new contexts, and persists between tasks, representing a significant advancement in AI systems.

In the rapidly evolving landscape of AI, organizations often lack a clear maturity model to assess their progress and capabilities, unlike established frameworks for cloud transformation and DevOps. This text discusses a proposed AI maturity model that spans five levels—Experimenting, Adopted, Observed, Governed, and Optimized—each representing stages of AI integration and sophistication within an organization. The model evaluates five dimensions: visibility and observability, evaluation and quality, cost governance, access and identity, and audit and incident response, providing a comprehensive framework for leaders to assess their AI practices. It emphasizes that the model serves as a diagnostic tool rather than a prescriptive target, encouraging organizations to close specific gaps that hinder progress, such as those in visibility or evaluation, rather than rushing to achieve the highest level. The text also highlights the role of the Pydantic stack in addressing various dimensions of the model and notes areas it does not fully cover, inviting leaders to engage in meaningful conversations about their AI strategies.

Developing AI applications often involves repetitive foundational work such as setting up authentication, real-time streaming, conversation persistence, observability, deployment, and CI/CD pipelines, which delays the implementation of business logic. The open-source full-stack-ai-agent-template aims to streamline this process by providing a CLI tool that generates production-ready applications using FastAPI, Next.js 15, and either Pydantic AI or LangChain for AI agents. This tool offers features like WebSocket streaming, JWT authentication, database setup with Alembic migrations, Redis caching, Logfire tracing, and Docker deployment with a single command. Pydantic AI is favored for its type safety and simpler model, while LangChain is recommended for projects requiring its extensive tool library and LangSmith tracing capabilities. The template also integrates Pydantic Logfire for enhanced observability, enabling developers to easily track and debug production issues by providing detailed traces and logs across multiple subsystems. This approach allows teams to bypass the repetitive scaffolding phase and concentrate on developing their unique AI solutions.

Pydantic AI v2 introduces a more advanced framework for developing AI agents by unifying various components into a single, composable unit called a "capability," which integrates instructions, tools, lifecycle hooks, and model settings. This approach allows agents to dynamically load tools and manage context, enhancing their adaptability and efficiency. The upgrade also emphasizes modularity, with core components remaining stable while additional features are developed in the Pydantic AI Harness, allowing for rapid innovation and community contributions. The release includes a shift in version policy, reducing the no-breaking-changes window between major versions from six months to three, to stay aligned with the fast-paced advancements in the field. The new capabilities, such as extended thinking, code mode, and tool search, demonstrate the framework's flexibility, while the integration of instrumentation allows agents to self-assess and suggest improvements. The system's design facilitates customization and scalability, enabling users to tailor agents to specific tasks while maintaining a stable core infrastructure.

User feedback highlighted the demand for skills, specifically progressive disclosure, in AI models, which led to the development of on-demand capabilities to address inefficiencies associated with loading all capabilities upfront. This approach, implemented in Pydantic AI, involves marking capabilities with `defer_loading=True` and using an ID to defer loading until necessary, reducing unnecessary input token costs. The model starts with a catalog of available capabilities and loads them only when required, optimizing resource use. The Pydantic AI framework facilitates this by allowing capabilities to be structurally gated, ensuring they only trigger when loaded, which reduces context and associated costs. Pydantic Logfire provides monitoring to track which capabilities are actually loaded, offering insights and savings, with new users receiving a $10 starter credit for inference through the Pydantic AI Gateway.

A multi-agent workflow in CrewAI, which used to take 90 seconds to complete, now takes four minutes without any apparent issues, prompting an investigation using a new metrics browsing feature that simplifies the process of identifying performance degradations. This feature allows users to select a namespace and a metric without needing SQL, providing visual breakdowns to quickly pinpoint issues, as demonstrated by tracing a retrieval latency issue back to a fragmented index that, once reindexed, restored the workflow to its original speed. The system groups metrics by namespaces like http or system and offers a seamless transition from discovery to detailed SQL analysis, ensuring users can efficiently identify and resolve performance problems without learning new query languages. The Metrics explorer is available for all Logfire projects, with precautions for avoiding double-counting in host metrics and a reference guide for automatically-collected metrics, and the platform also offers a free tier for new users.

A document classifier built on the Anthropic SDK is consistently failing at 2am due to out-of-memory (OOM) issues, despite no changes in code, model, or traffic. Investigation reveals that while the classifier's memory usage peaks at 95% just before failure, the increase begins prior to the job's start, indicating another process is consuming resources. Specifically, a Postgres vacuum job spikes disk I/O and contributes to the memory load. Utilizing OpenTelemetry (OTel) host metrics, the issue is diagnosed by observing CPU, memory, and disk activity, allowing for adjustments such as rescheduling the vacuum and capping memory usage, which resolves the problem. The system integrates various AI and monitoring tools, facilitating comprehensive diagnostics by linking host metrics and trace data, and employs a non-proprietary OTel collector to gather and display host metrics efficiently across platforms like Kubernetes. The solution demonstrates the importance of cohesive monitoring tools in identifying and resolving system performance issues, and offers an accessible setup for those using OpenTelemetry, with an invitation to try the free tier of Logfire for enhanced monitoring capabilities.

The Vercel AI SDK chatbot deployed on Google Kubernetes Engine (GKE) is experiencing intermittent 500 errors, with about 8% of requests affected following a recent deployment. This issue is attributed to a pod entering a restart loop due to an Out of Memory (OOM) error, caused by a new image that doubled memory usage. The monitoring solution described provides a comprehensive view of the Kubernetes cluster, allowing users to identify problematic pods, nodes, and workloads through sortable metrics. This system integrates with OpenTelemetry to correlate pod metrics with application traces, offering a unified interface to diagnose and address issues efficiently. The setup involves using specific OpenTelemetry collectors and processors, which can be configured through an intuitive setup process, allowing for quick rollback and adjustments to fix errors and complete deployments successfully. The platform supports seamless navigation between cluster metrics and application traces, enhancing the observability and troubleshooting capabilities within Kubernetes environments.

During a Black Friday scenario, a LangGraph workflow experiences timeouts on 8% of requests due to a slow service among 14 being called, with Redis identified as the culprit. The Pydantic Logfire platform, equipped with a Services view and a topology graph, efficiently navigates through service traces to pinpoint issues, illustrating how Redis calls took 11 seconds and caused delays. The platform enables users to seamlessly trace requests, errors, and latencies across services, offering a unified interface that integrates service maps into the live view for effective troubleshooting. This approach allows teams to quickly identify and resolve bottlenecks, like a Redis cache warming failure, restoring normal operations with just a few clicks.

In the evolving landscape of observability, traditional service-oriented approaches are inadequate for handling the complexities introduced by agents that call and operate on services, leading to unique failure patterns not captured by existing tools. Agents exhibit characteristics of distributed systems, including non-deterministic execution paths and volatile costs, as well as dependencies that can shift unpredictably, posing significant challenges for latency and data governance. As a result, a new class of observability tools is emerging to address these issues, with AI-observability and infra-observability vendors offering partial solutions, but none providing a comprehensive view until Logfire's product, which integrates all necessary observability features under one platform. This product offers a unified experience with capabilities like managed prompt variables, systematic testing, data loss prevention, and seamless integration with various frameworks through OpenTelemetry, thus enabling users to effectively monitor and optimize the performance and compliance of their agent-driven systems.

Logfire has simplified the process of querying telemetry data by allowing users to write SQL queries for metrics without dealing with the complexities of exponential histogram calculations. Previously, querying histogram metrics required extensive manual coding, but now users can perform the same operations using concise SQL queries, thanks to a unified 'value' column and a set of 'metric_*' functions that automatically handle the complexities of different metric types. This update not only reduces the time and potential errors involved in writing queries but also optimizes the performance of dashboards and alerts. The new system enhances both human and AI interactions with the data by simplifying the syntax and improving the speed of query execution, while maintaining compatibility with existing queries through backward compatibility. This advancement allows for more efficient data analysis and visualization in Logfire's platform, making it easier to obtain observation percentiles and other statistical insights from telemetry data.

Pydantic AI Gateway serves as a crucial operating layer for AI usage, offering a centralized platform for provider access, API keys, routing, and cost controls, thereby addressing the prevalent concern of cost management in AI projects. Logfire incentivizes new organizations to upgrade to its Team or Growth plans by offering a one-time $10 credit for using built-in provider services through the Gateway, which streamlines integration by automatically applying the credit without requiring additional steps. The Gateway allows requests to pass in their native format and integrates server-side limits at various levels, ensuring seamless use of new provider features and comprehensive telemetry integration with Logfire for enhanced cost visibility and observability. Many teams are leveraging the Gateway and Logfire together for these insights, even before deploying Pydantic AI, highlighting the Gateway's utility in providing actionable insights into AI usage without necessitating changes to existing frameworks.

AI spending within organizations is increasingly resembling headcount costs rather than traditional Software as a Service (SaaS) expenses, leading to governance challenges. Unlike flat subscription services, AI expenditure scales unpredictably with usage, and lacks clear ownership as multiple teams utilize AI tools without a centralized point of accountability. This lack of clarity can result in finance teams only noticing the extent of AI spending when it becomes substantial, often exceeding the cost of hiring a senior engineer. Effective governance of AI expenditure requires observability to track the purpose and outcomes of AI runs, which involves engineering efforts to generate actionable insights for financial management. The Pydantic AI Gateway and Logfire systems offer solutions by providing visibility and control over AI spending, allowing organizations to assess whether the costs are justified and to shape future expenditures strategically. As inference costs stabilize and market pressures increase, the urgency to address these governance issues becomes more pronounced, highlighting the need for integrated solutions that bridge financial and engineering perspectives to maintain coherence and control over AI-related expenditures.

The text delves into the intricacies of using Python's Abstract Syntax Tree (AST) module for parsing, understanding, and manipulating Python code, highlighting its potential in tackling complex tasks and its relevance with the advent of AI. The author explains how the AST can be used to backport modern Python type hint syntax to older versions, showcasing this through the eval-type-backport library. Several code snippets illustrate the process of transforming ASTs, such as replacing the | operator with safe_or to maintain compatibility across Python versions. The article also emphasizes the value of metaprogramming skills in enhancing one's career, noting the author's own experiences with companies like Grist and Pydantic. Furthermore, it argues that as AI increasingly automates routine code, advanced skills like metaprogramming will become more crucial, especially for AI agents that can execute Python code to improve their functionalities. This exploration of AST and its applications is part of a broader series on Python metaprogramming, aimed at deepening the reader’s understanding and capabilities in Python, with future posts promising insights into import hooks, tracing, and more.

AI observability platforms address the unique challenges of monitoring AI systems, which traditional application monitoring cannot fully capture, by providing comprehensive insights into the full execution of AI requests. These platforms are crucial for identifying silent quality failures, understanding agent behavior, managing runaway costs, and pinpointing root causes outside the model, such as slow database queries or rate-limited APIs. They offer benefits like faster debugging, measurable quality assessments, cost control, and confidence in shipping AI changes, all by integrating evaluation and tracing capabilities in one platform. Key features to consider in an AI observability platform include AI-native tracing, integrated evaluation, full-stack depth, open standards for portability, polyglot coverage, queryable data, predictable pricing, and scalability. Among various options, Pydantic Logfire stands out for its AI-native and full-stack tracing capabilities, open standards, and transparent pricing, while other platforms like Langfuse, LangSmith, Arize AI, Braintrust, and Datadog offer specialized features catering to different needs and preferences.

The text examines the importance of a "harness" in the development and operation of AI agents, emphasizing that while models are crucial, they are not the sole component for creating reliable agentic services. A harness acts as a governing layer that integrates various tools, coordinates tasks, ensures safety and fallback measures, and maintains a durable state that can be inspected and reused. This concept is pivotal for long-horizon AI systems, enabling them to operate efficiently and transparently within an organizational framework. The Pydantic stack, specifically through its Logfire platform, exemplifies this approach by providing a comprehensive runtime surface that unifies model calls, tool interactions, and application data, offering observability, governance, and adaptability across different workflows. The text argues that the future of agentic systems will rely on these harnesses to ensure that AI agents can perform meaningful work that is understandable and trustworthy to humans, rather than relying solely on the capabilities of individual models.

The author discusses their highly customized TODO app, which, despite its personal focus and complex features, serves as a learning tool. The app incorporates advanced technologies like CRDTs, bloom filters, and immutable trees, and has evolved to include an MCP interface for SQL-based querying, leveraging the DataFusion SQL engine. This approach transforms the app’s data model from a tree structure into a relational format, allowing for sophisticated SQL operations such as filtering, aggregation, and recursive queries. The integration of user-defined functions simplifies complex tree operations, and the app supports both reading and mutating data through SQL queries. This innovative setup enables efficient data interaction with minimal interface design, showcasing the app's evolution into a powerful tool for exploring modern data processing techniques.

In the guest post by Harald Tryti Rieber, co-founder and CTO at Atlas, the focus is on the significance of AI observability in enhancing the efficiency and reliability of agent products. While AI agents can automate and accelerate tasks, the real value lies in the engineering frameworks built around them, particularly observability, which allows for monitoring and understanding both the code and user interactions in production. Rieber explains how Atlas uses OpenTelemetry and MCP to maintain transparency and accountability in their AI operations, emphasizing the importance of context-rich data to make AI insights actionable. He highlights the utility of two simple skills, Logfire query and customer insight, which help streamline data queries and user behavior analysis, respectively. However, he cautions that the effectiveness of AI agents depends largely on the quality and completeness of the underlying data, and stresses that robust data engineering is essential for maximizing AI potential. Rieber invites others to share their experiences, suggesting that meaningful advancements often come from practical, understated solutions rather than sensationalized claims of rapid success.

The integration of Pydantic Codex plugins, particularly the Logfire Exporter and Logfire, enhances the debugging process by allowing Codex to directly interact with application telemetry, thus closing the gap between the agent's investigation and the application's performance data. The Logfire Exporter sends Codex session events to Logfire, while the Logfire plugin enables Codex to query logs and metrics, facilitating a seamless workflow where Codex investigates app failures and captures its investigative process in Logfire. This setup eliminates the need for manual data transfer, allowing Codex to autonomously query for exceptions and errors, streamlining the debugging loop by enabling a unified view of both the agent's actions and application behavior within the same telemetry framework. This integration leverages OpenTelemetry standards to create a cohesive environment where the agent can efficiently debug applications, providing a more efficient and less error-prone debugging process.

Bill Easton has been appointed as the Head of Product at Pydantic, bringing extensive experience in observability, security, and developer experience to the role. Previously, he led the product organization at Elastic focusing on OpenTelemetry, and held significant positions at Amazon and Verve Industrial Protection, where he developed security and observability solutions for critical infrastructure. Easton is familiar with the Pydantic ecosystem as a maintainer of FastMCP and a contributor to Pydantic AI, having actively engaged with the community and tools. His expertise in building scalable products aligns with Pydantic's mission, making him an ideal fit for the company, as highlighted by Samuel Colvin, the Founder and CEO of Pydantic.