September 2023 Summaries

Building a Retrieval-Augmented Generation (RAG) application for Zendesk Support using Verba and dlt enables companies to get insights from their internal knowledge data, making it easier for everyone in the company to find the right information at the right time. With Verba, users can ask questions and receive relevant answers based on proprietary data, while dlt simplifies the process of loading data from various sources, including CRM data. The application uses Weaviate as a vector database and OpenAI models for generating answers. By following a step-by-step guide, developers can set up Verba, install dlt with Zendesk source, create a pipeline to import data from Zendesk, load data into Verba, and ask Verba questions to retrieve relevant information. This application demonstrates how RAG technology can be applied to real-world problems, enabling companies to improve their knowledge management and decision-making processes.

A hiring manager's perspective on writing a data engineering CV for Europe and America is that it should demonstrate a good understanding of the role and what is expected, while also showcasing relevant skills and experience. A cover letter should be written to show motivation and why the candidate would be a good fit for the company, without being too generic. The CV itself should have a clear structure, with sections on bio, image, values, motivation, mission, skills, job experience, education, projects, and courses. Relevant skills should be listed first, and relevant technologies or tools mentioned in previous jobs can help personalize the list. Job experience should focus on achievements and takeaways, rather than just listing responsibilities. The hiring manager is looking for clear communication, better skills, and more applications, with opportunities depending on location and job market.

Dumpster diving for data: The MongoDB experience` discusses the challenges of extracting data from MongoDB and making it available in a SQL store. The article highlights the difficulties around ingesting data into a SQL database due to MongoDB's flexible schema, document-based storage, and potential data inconsistencies. It explores the requirements that drive this process, including business user access, ecosystem of integrations and tools, standardization for consistency, and data transformation & modelling capabilities. The article also delves into the differences between BSON and JSON, and how MongoDB can be solved by converting BSON to JSON. Additionally, it discusses the importance of data typing, cleaning, and unpacking nested JSON into tables or keeping it as JSON. Finally, it introduces a tool called `dlt` that helps load Mongo data into SQL databases, making the process more efficient and robust.