July 2024 Summaries

The Mixture of Experts (MoE) architecture is a machine learning framework that utilizes specialized sub-networks called experts to optimize model efficiency and performance. MoE models consist of multiple smaller neural networks, each focusing on specific tasks or data subsets, with a gating network directing input to the most appropriate expert. This approach reduces computational costs, enhances resource usage, and improves model performance by only activating relevant parts of the model for each input. The MoE architecture has several benefits over traditional neural networks, including increased efficiency, scalability, and specialization. However, it also presents challenges such as increased complexity and more complex training procedures. Applications of MoE models include natural language processing, computer vision, and speech recognition.

WebSockets and Session Initiation Protocol (SIP) are crucial technologies for facilitating real-time data exchange and communication in modern applications. WebSockets enable bi-directional, full-duplex communication between a client and server, while SIP is used to establish interactive communication sessions such as phone calls or video meetings. Both protocols play a significant role in developing performant applications that rely on real-time communication. Symbl.ai's conversational intelligence capabilities can be integrated with WebSocket and SIP through its Streaming API and Telephony API, respectively. These integrations allow developers to extract valuable insights from messages, calls, video conferences, and other interactions by leveraging the powerful large language model (LLM) that powers Symbl.ai's solutions, Nebula.

LangChain is an LLM chaining framework that streamlines the development of end-to-end large language model applications. It offers a comprehensive library of components designed for seamless integration and rapid prototyping. Key benefits include its expansive library, modular design, ability to create context-aware applications, and numerous built-in integrations. LangChain's main components include chains, document loaders, text splitters, retrievers, embedding models, vector stores, indexes, memory, prompt templates, output parsers, agents, and models. The framework can be used to build various applications such as knowledge bases, FAQ systems, recommendation systems, and onboarding/training assistants.

Fine-tuning a large language model (LLM) is the process of taking a pre-trained base LLM and further training it on a specialized dataset for a specific task or knowledge domain. This allows organizations to leverage existing AI development work and create personalized LLMs without having to train one from scratch, saving time and resources. Fine-tuning an LLM can be beneficial in various ways, including increased task or domain specificity, customization, and reduced costs. One potential use case for fine-tuned LLMs is customer service, where they can power chatbots, perform sentiment analysis, and generate content such as call summaries and key insights. Fine-tuning an LLM involves installing libraries, downloading a base model, preparing fine-tuning data, setting hyperparameters, establishing evaluation metrics, and fine-tuning the base model. Common pitfalls when fine-tuning an LLM include catastrophic forgetting, overfitting, underfitting, difficulty sourcing data, time requirements, and increasing costs.

AI agents are poised to dramatically increase the adoption of AI applications, with the market for AI agents projected to reach $110 billion by 2032. An AI agent is an application or system capable of executing a given task without direct human intervention. Multi-agent systems can be created by connecting two or more AI agents, allowing them to collaborate on more complicated tasks. Building a multi-agent chatbot involves choosing a framework, choosing an LLM, installing packages, configuring the agents, creating and initializing the agents, and implementing conversation logic. Use cases for multi-agent chatbots include Q&A, customer support, customer service, semantic search, sentiment analysis, content generation, and education assistant.

Large Language Models (LLMs) have shown impressive capabilities in sentiment analysis and emotion detection. However, their learning and interpretation of language differs significantly from human language acquisition. This study aims to explore whether LLMs trained with multimodal features effectively utilize those features when processing data from a single modality. The research focuses on comparing general LLMs against specialized LLMs on multimodal data, specifically Llama-2-70B and its fine-tuned version for human conversation data (Llama-2-70B-conversation). The study uses deceptive communication as a challenging use case to evaluate the multimodal transfer of skills in LLMs. The results show that conversation+text models outperform unimodal text models in identifying deceptive communication, such as sarcasm, irony, and condescension. Additionally, emphasizing conversational features in prompts yields mixed results, with slight improvements in accuracy and precision but a decline in recall. The findings suggest that multimodal feature transfer occurs in LLMs, improving their performance on specific tasks that may require multimodal training. Further research is being conducted to investigate the effect of other modalities associated with human conversation data on the feature transfer phenomenon in LLMs and overall accuracy of challenging tasks for today's large language models.

Symbl.ai has introduced new customized scoring features to its Call Score API, a GenAI-powered evaluation tool that assesses human and AI agents using text and voice signals. The latest addition of 'Custom Criteria' and 'Scorecards' allows businesses to define their own evaluation criteria and build comprehensive scorecards for consistent evaluations across both human and AI agents. This helps ensure high quality interactions and improved customer experiences by providing in-depth visibility into the performance of customer-facing teams. Call Score API is designed to cater to various use cases across different sectors, including recruiter efficiency, patient engagement in telehealth, product pitch assessment, sales process alignment, and cold call analysis.

The new enhancements to Symbl.ai's Call Score API include 'Custom Criteria' and 'Scorecards', allowing for precise evaluation tailored to specific industries such as sales, customer service, healthcare, retail, finance, education, hospitality, legal, real estate, media, and entertainment. Custom Criteria enables users to develop detailed assessment criteria while Scorecards combine various criteria for comprehensive evaluations. The API now supports Streaming API for real-time conversations and Webhooks for call score status updates, improving overall customer experience and streamlining conversation processing.