June 2026 Summaries

PixelGolf, an internal project still under development, is designed to provide golfers with detailed swing analysis by aggregating data from swing video frames. The process involves using MediaPipe for per-frame analysis and leveraging Pixeltable's User Defined Aggregator (UDA) for effective data aggregation, which has proven to be the most challenging aspect of the project, rather than pose detection or coaching through large language models. Despite experimenting with various methods like user-defined functions (UDFs) and queries, the team found that using UDA in combination with group_by queries is the most efficient approach for aggregating metrics across multiple frames into a single summary. However, challenges remain in implementing a fully declarative rollup of aggregated data onto a parent table, an aspect the team hopes to improve with future updates to Pixeltable. The blog post shares insights and lessons learned from the development process, emphasizing the importance of choosing the right Pixeltable primitives for different aggregation tasks in sports video analysis.

Convex and Pixeltable provide distinct yet complementary solutions for full-stack and AI-focused developers, respectively, by offering reactive TypeScript backends and declarative data planes. Convex excels in coordinating TypeScript full-stack applications with its document-relational database, server functions, and client libraries that enable live data updates through WebSockets. In contrast, Pixeltable shines as an AI data layer, integrating tables, computed columns, and embedding indexes to manage multimodal storage and AI transformations, with incremental updates ensuring data freshness. The post explores mapping Convex primitives onto Pixeltable using a help-desk app, highlighting how production systems often benefit from using both for optimal application coordination and AI data handling. Through detailed examples and feature comparisons, the text outlines how each platform handles database layers, server functions, and client layers, emphasizing their respective strengths for live UI updates and AI retrievals, while also offering guidance on when to use each or both in unison.

At the Data + AI Summit 2026, Databricks introduced LTAP (Lake Transactional/Analytical Processing) with Lakebase, a Postgres-compatible transactional layer that transforms data into open columnar formats like Iceberg, aiming to unify analytics and transactions for AI agents without the traditional ETL tax. While the engineering feat of achieving approximately 25,000 transactions per second on open formats is significant, LTAP is essentially a managed change-data capture (CDC) system with a sync delay, rather than the hybrid transactional/analytical processing (HTAP) it was marketed as. Although this dual-engine approach may be pragmatic for enterprise tabular data, AI developers face additional challenges like managing multimodal data that cannot be addressed by LTAP alone. Pixeltable emerges as a complementary solution for multimodal AI workloads, focusing on declarative, incremental compute pipelines that eliminate mirror lag, thus offering a more integrated approach for managing complex data types such as video, audio, and model outputs. While LTAP addresses the OLTP/OLAP boundary for tabular data, Pixeltable provides the necessary infrastructure for multimodal write paths, highlighting the importance of choosing the right architecture for specific AI application needs.

Microsoft has open-sourced pg_durable, a PostgreSQL extension designed to facilitate durable, fault-tolerant workflows within the database itself, eliminating the need for external orchestrators or complex status-table setups. This extension aligns with the principle of bringing compute close to data, similar to what Pixeltable does with multimodal data like video and audio. pg_durable allows for the definition of workflows as graphs of SQL steps, with execution resuming from the last durable checkpoint in case of failures, thus simplifying development by reducing reliance on cron jobs and external orchestrators. This approach exemplifies a broader architectural shift toward integrating execution state with data, a concept also embraced by AI infrastructure like Pixeltable, which targets multimodal workloads through declarative, incremental compute close to diverse data types. By embedding orchestration within Postgres, pg_durable signals a mainstream acceptance of this paradigm, while Pixeltable complements it by addressing the needs of complex AI applications that require handling diverse media types and advanced model inferences beyond SQL's scope.

The May and June Pixeltable release cycle, covering versions 0.6.1 through 0.6.5, introduced a range of enhancements to support developers in building multimodal AI applications in Python by managing data with tables that execute model calls, transform media, and index embeddings. Key updates focused on improving the operability and inspection of multimodal pipelines, including a first-class CLI and local dashboard for catalog management, production hardening for threaded serving, and integration with vLLM and Iceberg export. The CLI has become the primary tool for daily operations, providing capabilities such as listing directories, inspecting schemas, and debugging errors, while the local dashboard offers a visual interface for catalog exploration. These releases also emphasized improving reliability under concurrent loads, with enhancements like thread-safe tables and parameterized query functions. Additional integrations with vLLM and Google Gemini 2.5 models, as well as support for Iceberg exports, extend Pixeltable’s functionality within larger analytical ecosystems. Various smaller fixes and compatibility updates were also included, ensuring that workflows run smoothly and efficiently.

Pixeltable 0.6.5 introduces the export_iceberg() function, enabling the streaming of table or query results into an Apache Iceberg table, thus enhancing its role as an AI data infrastructure layer by supporting multimodal workflows with storage, computed columns, embeddings, versioning, and lineage. This feature fills the gap for analytics teams needing curated outputs in a lakehouse format suitable for SQL dashboards, feature stores, or warehouse joins, by providing open table format semantics like ACID commits, schema evolution, time travel, and catalog-backed tables. Pixeltable supports various export paths including export_sql(), export_lancedb(), export_parquet(), and export_csv(), and now with export_iceberg(), it facilitates seamless data handoff to Iceberg-compatible analytics stacks like Spark, DuckDB, and Trino. The export process leverages PyArrow for memory-efficient streaming, allowing for batch size control and schema overrides to ensure compatibility and flexibility in downstream analytics workflows. The addition of this functionality aligns with existing practices of using cloud blob storage for raw assets and analytics tables for derived tabular artifacts, and further documentation including an Iceberg cookbook is underway to assist users in maximizing the utility of these features.

Pixeltable 0.6.5 introduces native integration with vLLM, enhancing its capacity to run HuggingFace models at high throughput within computed columns while maintaining key features like declarative pipelines, versioning, and per-row error management. This integration specifically targets GPU throughput constraints, allowing for efficient batch processing and large-scale document classification, summarization, or evaluation tasks. The system supports incremental computation, ensuring only new or altered rows are processed, and provides a version history to compare outputs across changes. While Ollama supports local development and quick experiments on modest hardware, and llama.cpp caters to quantized models on CPUs or Apple Silicon, vLLM offers robust batch inference capabilities on GPU clusters. Additionally, vLLM loads and caches models from HuggingFace, facilitating efficient reuse in computed-column evaluations, with flexible options for engine and sampling parameters.

Pixeltable offers a comprehensive local dashboard designed for visually exploring catalog data, providing a read-only web interface that complements its CLI and Python SDK. The dashboard allows users to navigate through tables, views, and snapshots, offering features such as global search, in-table row filtering, and inline media rendering for images and videos. Users can inspect data with options like paginated views, lineage graphs, and version history, while maintaining data integrity by ensuring updates and schema changes occur outside the UI. The dashboard provides functionality for debugging errors, exploring dependency graphs, and exporting data, with tools for both visual inspection and scripted analysis. It supports troubleshooting through command-line operations and utilizes keyboard shortcuts to enhance navigation and usability, ensuring efficient data exploration and management.

The concept of an agent is redefined as the durable history of interactions, including user inputs, model outputs, tool calls, and results, rather than being tied to the model, runtime, or specific processes executing tasks. This history, when properly maintained, allows for reliable recovery from crashes, scalability, branching experiments, multi-user workflows, and seamless migration between different models or providers. The session log, paired with a session definition, forms the complete state of an agent, allowing any process to pick up and continue a task from where it left off. By treating this log as the primary record, similar to a database's write-ahead log, agents become more reliable and adaptable, with their actions and decisions being traceable and auditable. The log is seen as a foundational element, with tools like Pixeltable offering a structured, versioned, and queryable system for storing session data, thus enabling better state management, auditability, and experimentation without being tightly coupled to specific execution frameworks.

Pixeltable 0.6.5 introduces a robust command-line interface (CLI) called pxt, designed to streamline catalog operations and service deployment. Users can define schemas and computed columns in Python and leverage the CLI to perform tasks such as listing tables, inspecting rows, and managing table versions without additional application code. The CLI facilitates fast operations through a local daemon and offers machine-readable JSON output for automation and AI integration. It also supports HTTP endpoint creation using FastAPI with pxt serve, eliminating the need for extensive boilerplate code. The CLI's functionality includes inspecting, querying, and managing tables and directories, with an emphasis on automation-friendly features like JSON output. This tool operationalizes the Python SDK, making it easier to inspect, debug, script, and serve data catalogs directly from the terminal, thereby enhancing workflows for agents and coding assistants.

"Make building multimodal AI apps dead simple" is a promise to streamline the complex process of developing AI applications by consolidating disparate components like storage, orchestration, and data retrieval into a single developer experience, allowing developers to focus on application logic rather than infrastructure. The approach hinges on integrating AI transformations directly into the schema, ensuring that tasks such as unified storage, automated orchestration, effortless retrieval, and a seamless developer experience occur effortlessly behind the scenes. Pixeltable embodies this concept by allowing media to be treated as a first-class column type within a cohesive environment, automatically managing data lineage, versioning, and embeddings. This method eliminates the need for separate systems to maintain vector databases or orchestrate ETL processes, enabling incremental recomputation and offering a consistent development experience across local and production environments. The system is designed to simplify the multimodal AI data layer, though it acknowledges that real-time streaming and high-throughput workloads remain areas for future development.

In the landscape of modern software development, coding agents have drastically reduced the cost of writing code, making it almost negligible and shifting the expense to token usage, which is rapidly increasing. This shift necessitates an efficient data infrastructure that prevents redundant data processing and leverages open-source models to manage costs effectively. While coding agents can generate extensive code quickly, they lack the capability to architect a comprehensive and efficient data layer, which is essential for maintaining scalable and reliable systems, especially with complex data like video. The emergence of platforms like Pixeltable addresses this need by offering a data infrastructure harness that integrates best practices into a few declarative primitives, ensuring production-ready and token-efficient code. As software development transitions from expensive to build but cheap to run, to the opposite, companies must focus on optimizing their data layers and infrastructure to maintain efficiency and cost-effectiveness, rather than relying solely on the rapid prototyping capabilities of coding agents.