October 2024 Summaries

Azure customers now have general access to OpenAI's GPT-4, which remains a leading LLM despite the emergence of alternatives like Claude and Llama 2. While GPT-4 is praised for its detail retention and instruction-following capabilities, privacy and speed concerns have previously deterred some users. Azure OpenAI offers significant speed improvements over OpenAI's public API, though it imposes stricter rate limits and requires careful management of API versions, which can affect prompt responses. Azure's platform also includes content filtering by default, which can be customized. Moving to Azure entails regional deployment and naming considerations, and while startups benefit from generous credits, users encounter challenges such as wait times for model updates and fine-tuning limitations. Despite Azure's complexities, it provides a robust environment for deploying and managing AI models across regions, offering advantages in speed, credits, and user experience for multi-region deployments. Azure's rate limits and deployment nuances necessitate strategic planning, and while fine-tuning options are limited, Azure remains a preferred platform for many despite competition from AWS and Google Cloud.

In 2023, discussions around AI safety and regulation were prominent, particularly with the anticipation of the UK's AI Safety Summit and concerns about the implications of large language models (LLMs) like GPT-4. Contrary to sensationalized media narratives, actual AI safety incidents are not as rampant as suggested, with only a modest increase in reported events and many incidents being unrelated to AI technology itself. Generative AI, while often misunderstood as being autonomous, functions as an advanced probabilistic tool that excels in generating content across various media. The year also highlighted vulnerabilities in AI systems, such as adversarial suffixes and prompt injection, but these received minimal attention compared to broader fears about AI's potential societal impacts. As AI systems become more capable of performing tasks traditionally done by humans, there is anticipation of significant shifts in the workforce, particularly in administrative roles, prompting discussions on economic structures like universal basic income. Despite regulatory efforts and concerns about AI's influence, leading AI companies continue to prioritize alignment and safety, while debates persist on how best to integrate AI advancements into society responsibly.

Vector databases (Vector DBs) have gained significant attention, especially in 2023, and are considered crucial for certain products despite some companies potentially being more feature-centric. The author shares their journey of building Klu, a platform using Supabase and pgvector, which has evolved from a few models and scripts to handling over 500,000 weekly requests with a custom vector engine. While acknowledging that their system isn't the fastest, the author emphasizes prioritizing quality data retrieval and user experience over mere speed, arguing that latency is less critical when compared to the latency of language models. The narrative underscores the importance of focusing on the speed of learning and building, rather than adhering strictly to conventional methods, highlighting the flexibility and support offered by Supabase in contrast to other platforms like Pinecone. In Part 02, the author plans to discuss internal benchmarks, noting that neither Supabase nor Pinecone is the fastest, hinting at potential alternatives like Chroma.

Prominent AI leaders initially called for a pause in developing advanced AI systems beyond GPT-4, but breakthroughs have continued, particularly in smaller open models. Concerns persist about AI's potential to generate harmful or false information, and the lack of understanding of how these models work internally contributes to these fears. However, AI labs claim significant progress in controlling outputs using techniques such as response blocking, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF), which align large language models (LLMs) like GPT-4 and Claude 2 with human values. Companies like Anthropic, Microsoft, and OpenAI are employing advanced content filtering systems to mitigate harmful content, while efforts to specialize large models for specific applications continue. The development of AI safety measures, including industry collaborations and RLHF techniques, underscores ongoing improvements in AI reliability and safety, challenging the notion that we lack control over these technologies.