How to Monitor DataStax-Powered RAG Applications with Fiddler
Blog post from DataStax
Fiddler AI and DataStax have collaborated to enhance retrieval-augmented generation (RAG) based large language model (LLM) applications by integrating Fiddler's AI Observability platform with DataStax's Astra DB. This integration aims to optimize the performance, accuracy, safety, and privacy of GenAI applications by utilizing Astra DB's real-time vector search capabilities and Fiddler's LLM application scoring powered by Trust Models. The technical process involves onboarding applications like DataStax's WikiChat to the Fiddler environment, allowing for the monitoring of prompts and responses across various trust-related dimensions such as faithfulness and PII leakage. Developers can use tools like Next.js to ingest data into Fiddler, enabling real-time detection and diagnostics of issues such as prompt injection attacks and toxic responses. By tracking relevant LLM metrics, developers can align these insights with business KPIs and explore further enhancements through resources like Fiddler's AI Chatbot guide and Astra DB's free offerings.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| LLM | 19 | 4,030 | 486 | 147 | +1% |
| RAG | 11 | 1,966 | 260 | 82 | -21% |
| Observability | 5 | 1,798 | 331 | 106 | +34% |
| Vector Search | 4 | 3,701 | 290 | 90 | +59% |
| Real-time | 3 | 4,377 | 976 | 225 | +49% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.