September 2024 Summaries

Preprocessing is an essential step in building a Retrieval Augmented Generation (RAG) pipeline, accounting for about half of the project's workload. It involves preparing and indexing data so that the RAG system can generate accurate answers. The process includes examining and extracting data, cleaning it, chunking it into optimal lengths, adding metadata, and finally indexing it. Advanced techniques such as Named Entity Recognition (NER), language classification, semantic chunking, and multimodal processing can be incorporated to customize the preprocessing pipeline for specific use cases. In production systems, distributed architectures and technologies like Kubernetes are used to manage high throughput and low latency requirements. deepset Cloud offers a comprehensive solution for indexing with its speed, flexibility, and ease of customization.

Retrieval augmented generation (RAG) systems are becoming standard for generative AI applications with large language models (LLMs). They improve LLM performance, reduce hallucinations, and dynamically expand the knowledge base. However, business leaders should consider customizing basic RAG setups to meet specific needs and gain a competitive advantage. Compound AI's modular approach allows additional components to be added to the basic setup for more sophisticated systems tailored to unique use cases. Customization strategies include query classifiers, hybrid retrieval, rankers, reference prediction, and advanced setups like Agentic RAG and GraphRAG.

The role of an AI engineer is distinct from traditional software engineers or data scientists, as it focuses on productizing AI advances and making them consumable through REST APIs. Key characteristics of an AI engineer include being well-versed in heterogeneous development environments, having a product-centric mindset, and thinking in terms of APIs. Prompt engineering has become part of this API mindset, with AI engineers needing to be aware of the specifics of prompt engineering per LLM. The role of the AI engineer is becoming increasingly important as it helps bridge the gap between AI/ML research, data engineering, data science and practical business applications.

Deepset, recognized as a leading AI startup in Germany, is collaborating with Meta's Llama Stack to enhance the development of sovereign, domain-specific AI solutions for enterprises, the public sector, and defense. This partnership aims to accelerate the adoption of customized AI across various industries, as highlighted by their inclusion in Sifted's 2025 "Rising 100" for B2B SaaS companies. Organizations such as Airbus, The Economist, and OakNorth have chosen deepset for its advanced AI capabilities, demonstrating the company's growing influence and success in the AI sector.