The article details the journey of Jimmy, a nuclear engineering company in France, in developing a Retrieval-Augmented Generation (RAG) system to efficiently search through vast amounts of complex technical documentation. Initially adhering to conventional best practices such as context-aware chunking and hybrid search, the team found these approaches underperformed in their specific context. Instead, simpler methods like naive chunking and dense-only search yielded better results, with AWS Titan V2 embeddings outperforming those from the MTEB leaderboard. The team emphasizes the importance of benchmarking under diverse conditions rather than relying on traditional benchmarks, highlighting that chunk size was not critical for document-level retrieval. They also discuss the choice of Qdrant for vector database storage and the use of Mistral OCR for PDF conversion, cautioning against AWS OpenSearch due to high costs. Overall, the article underscores that best practices in RAG systems should be tested and tailored to specific use cases rather than followed blindly.