March 2024 Summaries

James Murdza developed GitWit, an AI-driven online editor designed to facilitate the creation of React applications by automating complex coding tasks, especially for less experienced developers. To evaluate the performance of large language models (LLMs) within GitWit, Murdza is also developing ReactEval, an open-source benchmarking tool specifically for frontend applications. ReactEval addresses the challenge of assessing LLMs' effectiveness in generating code by automating the execution of numerous tests using E2B sandboxes, which simplifies the process of building and testing React applications in a web browser. By running hundreds of tests, ReactEval provides insights into LLMs' reliability and helps improve GitWit's performance, specifically in managing dependencies and code generation. GitWit has evolved from a tool that autonomously generates entire applications to a more interactive editor that allows users to contribute their skills alongside AI, making the development process faster and more iterative. Murdza is open-sourcing ReactEval and collaborating with other companies to enhance the tool's capabilities, while also planning to release demo videos and open-source components to further engage the developer community.

Rubric Labs, co-founded by Ted Spare, is an AI-focused software studio that collaborates with startups and financial firms to create AI-enabled products, while also contributing to open-source projects. One of their notable projects is Maige, an AI companion designed for codebase managers and open-source maintainers on GitHub. Maige operates through an event-driven architecture that integrates easily with repositories and automates tasks like labeling issues and reviewing pull requests. It utilizes a tech stack that includes OpenAI's GPT-4 Turbo, Weaviate for vector databases, and E2B sandboxes for executing AI-generated code. The project emphasizes open-source principles, offering users the ability to self-host, which aligns with the growing trend of software commoditization and community-driven development. Challenges in developing Maige include handling the non-deterministic nature of AI and ensuring security, while future plans focus on enhancing front-end development capabilities and improving code search functionalities. As AI technologies advance, Rubric Labs aims to leverage faster and more cost-effective models to expand Maige's capabilities.