March 2024 Summaries

Artificial Intelligence (AI) agents are applications or systems capable of executing tasks autonomously without direct human intervention. They work through a process of assigning, creating, or inferring an objective, which they then break down into a series of tasks and attempt to complete. AI agents can be applied in various industries such as healthcare, transportation, manufacturing, customer service, gaming, and more. The benefits of using AI agents include efficient automation, improved decision-making, reduced human error, increased availability, safety, cost savings, and scalability. As research into the applications of autonomous agents continues and their capabilities increase, they will play a larger role in various sectors.

Large language models (LLMs) have shown remarkable capabilities across various tasks but their performance in handling human-human interactions remains a question. Human conversations are complex and multifaceted, with interactivity, contextuality, adaptability, and emotional states as key characteristics. However, an analysis of LLM training data reveals that human conversation data is underrepresented, constituting only 0.0085% of the total data. This imbalance leads to artifacts that hinder a model's ability to effectively handle such underrepresented domains. Quantitative and qualitative analyses show that human conversations demand significantly longer attention distances compared to other data types, indicating the need for more robust modeling of long-term contextual relationships by models. To address this issue, Symbl has developed Nebula, an LLM specialized for human conversations, trained with a significant amount of conversation data and optimized to work best on such interactions.

Large language models (LLMs) are crucial for various applications, but selecting the best one requires considering several factors such as parameter count and performance on benchmark tests. Hyperparameters play a significant role in customizing LLMs to specific needs. They govern the training process of an LLM without becoming part of the resulting base model. Commonly used LLM hyperparameters include model size, number of epochs, learning rate, batch size, max output tokens, decoding type, top-k and top-p sampling values, temperature, stop sequences, frequency and presence penalties. Hyperparameter tuning is a process to find the optimal combination of these parameters for better LLM performance. Automated hyperparameter tuning methods like random search, grid search, and Bayesian Optimisation can streamline this process.

Large Language Models (LLMs) are becoming increasingly popular, and choosing the right one for your needs is crucial for the success of your generative AI strategy. One important aspect to consider when evaluating LLMs is their inference performance, which measures how quickly they generate responses. This guide explores LLM inference performance monitoring, including how it works, the metrics used to measure an LLM's speed, and how some popular models on the market perform. LLM inference involves two stages: a prefill phase where input tokens are processed and converted into vector embeddings, and a decoding phase where output tokens are generated one at a time until reaching a stopping criterion. The most important LLM inference performance metrics are latency and throughput. Latency measures how long it takes for an LLM to generate a response, while throughput provides a measure of how many requests the model can process or how much output it can produce in a given time span. Some challenges associated with measuring LLM inference include lack of testing consistency, different token lengths per model, and lack of data. To compare popular LLMs on these metrics, various benchmark tests have been conducted by organizations like Artificial Analysis, GPT For Work, and Predera. These tests provide valuable insights into the performance of different models under varying conditions. In conclusion, while inference performance monitoring is an important factor to consider when selecting an LLM, it should not be the sole determinant. Researching how a language model performs at various benchmarking tests can also help identify the best LLM for your specific needs.