September 2023 Summaries

Arcee has introduced a commercial product to enhance language models through domain adaptation, focusing on creating Domain Adapted Language Models (DALMs) that cater to specific business niches by employing a two-step process comprising domain pretraining and end-to-end retrieval augmented generation (E2E RAG). Domain pretraining ensures the language model's base generator is familiar with the target domain's vocabulary, using techniques to prevent the loss of general knowledge. The E2E RAG approach improves upon general retrieval models by training the retriever and generator models together, enhancing their contextual understanding. Arcee's platform offers open-source resources for community collaboration, alongside commercial tools and infrastructures for training and deploying DALMs at scale, ensuring minimal latency and optimal performance during inference. The platform supports private deployments within a company's VPC to safeguard proprietary data and intellectual property while providing dashboards for monitoring and automating training processes.