September 2023 Summaries

Deepgram introduces enhanced redaction options for customers submitting pre-recorded audio to their hosted endpoint. The advanced functionality allows users to select specific types of entities like locations, credit card numbers, or zodiac signs to be redacted from transcriptions, ensuring privacy and confidentiality. This feature is particularly useful for professionals in areas such as journalism, legal services, podcasters, video content creators, and others who need to maintain the accuracy and privacy of their content without sacrificing time. Deepgram's mission is to make working with audio data easier, more intuitive, and safer by providing maximum control over transcription output.

LegalBench is a collaborative legal reasoning benchmark designed to test the abilities of large language models (LLMs) like GPT-3 and Jurassic. Unlike other benchmarks, LegalBench is an ongoing project that anyone can contribute to. Its goal is not to replace lawyers but to determine the extent to which these systems can execute tasks requiring legal reasoning, thus augmenting, educating, or assisting them. The benchmark includes two types of tasks: IRAC reasoning and non-IRAC reasoning. LegalBench employs the IRAC framework to categorize and evaluate various legal tasks, including issue, rule, application, and conclusion tasks. It also includes non-IRAC tasks, referred to as “classification tasks.” The project is ongoing, with community contributors creating additional tasks according to the benchmark's guidelines.

The article discusses the use of AI-powered content moderation in social media platforms to combat online harm, such as graphic violence, hate speech, and harassment. It explains how autonomous content moderation systems use machine learning algorithms trained on large datasets to analyze and recognize patterns in language and classify content. The process is more complicated when audio and video elements are added, requiring speech-to-text conversion, contextual analysis, computer vision, generative adversarial networks (GANs), and optical character recognition (OCR). Sentiment analysis is also important for deciphering nuances in tone and context. The societal implications of AI content moderation include reducing mental health issues among human moderators and addressing the risk of replicating real-life biases and discrimination with AI-powered moderation models.

Mixture-of-Experts (MoE) is a technique in artificial neural networks that allows for efficient scaling of model capabilities without introducing significant computational overhead. Proposed in 1991, MoE adopts a conditional computation paradigm by selectively activating parts of an ensemble, or "experts," based on the data at hand. In recent years, MoE has gained popularity with the rise of large language models and transformer-based models due to their ability to handle complex datasets. The MoE architecture consists of dividing a dataset into local subsets, training expert models for each subset, using a gating model to interpret predictions from each expert and decide which expert to trust for a given input, and employing a pooling method to make a prediction based on the output from the gating network and the experts. In 2017, an extension of MoE suited for deep learning was proposed by Noam Shazeer et al., introducing the Sparsely-Gated Mixture-of-Experts Layer, which consists of numerous expert networks and a trainable gating network that dynamically selects a sparse combination of these experts to process each input. MoE has shown impressive results in domains like NLP and computer vision, but there is still much room for exploration and improvement in its design and application across various fields.

Deepgram introduces Nova-2, a next-generation speech-to-text model that outperforms alternatives in terms of accuracy, speed, and cost. Nova-2 is 18% more accurate than its predecessor and offers a 36% relative WER improvement over OpenAI Whisper (large). It delivers an average 30% reduction in word error rate (WER) over competitors for both pre-recorded and real-time transcription, with 5-40x faster pre-recorded inference time. Nova-2 is priced at $0.0043/min for pre-recorded audio, making it more affordable than other full-functionality providers. The model has been trained on a diverse dataset and offers improved entity accuracy, punctuation accuracy, and capitalization error rate compared to Nova-1. Deepgram's benchmarking methodology uses over 50 hours of human-annotated audio across various domains and compares Nova-2 with other prominent models in the market.

In 2023, the AI API Playgrounds have become a popular way for developers to experiment with various APIs and test their ideas without affecting live environments. These playgrounds offer rapid prototyping, skill enhancement, iterative collaboration, cost-effectiveness, and immediate feedback. Some of the best AI API Playgrounds in 2023 include EleutherAI's API Playground, Deepgram's AI Playground, HuggingFace's API Playground, and OpenAI's API Playground. Each playground offers unique features and benefits for developers and researchers at different experience levels.

The evolution of computing hardware has played a significant role in driving progress in Artificial Intelligence (AI) and Machine Learning (ML). Graphics Processing Units (GPUs), initially designed for rendering graphics in video games, have become game-changers due to their parallel processing capabilities that can be harnessed to accelerate AI computations. The positive correlation between the development of new computing hardware and advancements in AI/ML is evident, with GPUs, Tensor Processing Units (TPUs), and other specialized hardware accelerating these computations. The backbone of training and inference in AI/ML systems relies on robust hardware architecture to handle large datasets efficiently. Specialized hardware like Apple's M1 and M2 chips, Nvidia's A100 GPUs, and Google's TPUs are optimized for matrix math and tensor operations, enabling faster iteration through batches of training data compared to general-purpose CPUs. New optimizations in hardware architecture include the use of analog in-memory computing, phase-change memory, memristors, and optical systems. Software frameworks like CUDA by Nvidia, Ray by Anyscale, and Caffe2 help deploy and scale models across different hardware backends. Choosing the right hardware infrastructure is crucial for determining the speed and efficiency of model training and deployment. Factors such as model complexity, overhead, latency bottlenecks, and off-the-shelf vs. custom training should be considered when selecting hardware. The current shortage of GPUs has led to a bottleneck in AI innovation progress, particularly for smaller startups and researchers. This scarcity affects not only cost but also development velocity, with major cloud service providers grappling with oversubscription for Nvidia's latest GPU offerings. The GPU gold rush is on, and access confers a competitive edge to those who can secure it.

Prompt Engineering is the art of crafting effective prompts for large language models (LLMs) like ChatGPT to generate desired outputs such as text, images, or code. It involves using a combination of instructions, questions, input data, and examples in the prompt. There are various flavors of prompt engineering, including instruction prompting, persona pattern prompting, chain-of-thought prompting, self-consistency decoding, and prompting for image generation. By carefully crafting prompts, users can significantly improve the quality of AI-generated outputs.

Meta has introduced its latest large language model (LLM) project called Code Llama, which is designed to write code based on natural language prompts. This open-source model sets itself apart from other generative AI models with its permissive community license and positions itself as the foundation for a new wave of developer tools and coding assistants. Built upon Meta's Llama 2 model, Code Llama is trained on 500 billion tokens of code-heavy data and supports multiple programming languages. The model comes in various sizes and has capabilities such as code generation, code discussion, code completion and debugging, and support for multiple programming languages.

Language models (LMs) are designed to handle obscure or unknown words by using context clues from surrounding words. These models learn the likelihood of a word appearing based on its neighboring words, which is known as lexical semantics. However, LMs can struggle with out-of-vocabulary (OOV) words that were not part of their training data. One approach to handling OOV words is to replace them with the most likely word based on surrounding context. Another method involves breaking down words into smaller morphemes or subwords, which can help LMs better understand and predict the meanings of unfamiliar words. However, this approach may not be effective for all languages due to differences in morphological structures. To improve LMs' ability to handle OOV words, researchers recommend incorporating linguistic aspects beyond morphology into language models. This could involve developing separate tokenizers tailored to different language families and their unique morphological structures. By better capturing the complexity and diversity of human languages, LMs can more effectively handle unknown or obscure words in various contexts.

The evolution of speech synthesis has come a long way since its early days as a keyboard connected to a loudspeaker through circuitry. Speech synthesizers have evolved from simple machines producing vowels and consonants to sophisticated systems using deep learning algorithms for natural-sounding speech generation. Early inventors like Wolfgang von Kempelen, Charles Weathstone, and Alexander Graham Bell contributed to the development of speaking machines that could produce full words. The vocoder, developed by Homer Dudley in 1937, was a significant milestone in speech synthesis technology. In recent years, deep learning-based speech synthesis has emerged as a powerful tool for generating more natural and intelligible speech. Google's WaveNet and Tacotron 2 are examples of projects using neural networks to produce high-quality synthesized speech. Today, speech synthesis is widely used in voice assistants like Siri, Cortana, and Alexa, as well as in various applications such as text-to-speech systems for people with disabilities. The history and development of synthetic speech have been shaped by the contributions of many scientists and researchers over centuries, leading to advancements that continue to improve our interactions with technology.

Recent research has highlighted the potential risks associated with training large language models (LLMs) on synthetic data generated by other AI models. This practice, known as "model collapse," can lead to low-quality outputs and reinforce biases inherent in the synthetic data. The phenomenon of AI eating itself raises ethical concerns and poses a serious question for the future of AI development. Some potential consequences include hallucinations, machine scale security attacks, algorithmic biases, loss of human innovation and creativity, extinction risks, AI overlords and authoritarianism, and public information scarcity. To mitigate these risks, companies developing LLMs must carefully select their training datasets and utilize responsible AI development practices, prioritizing diversity in the research, development, and implementation stages.

The article discusses the top AI newsletters on Substack that readers should not miss in 2023. It highlights five newsletters: AI Snake Oil, The Algorithmic Bridge, The Gradient, SemiAnalysis, and What Did OpenAI Do This Week? Each newsletter is briefly described, with information about their content, target audience, and subscription options. The article also provides recommendations for readers based on their experience level in the field of AI.

The HumanEval dataset and pass@k metric have revolutionized how we measure the performance of LLMs in code generation tasks. HumanEval is a hand-crafted dataset consisting of 164 programming challenges, each with a function signature, docstring, body, and several unit tests. Traditional evaluation methods for generated code involved comparing the produced solution with the ground-truth code using metrics like BLEU score, which measure text similarity rather than functional correctness. The pass@k metric addresses this limitation by evaluating the probability that at least one of the top k-generated code samples for a problem passes the unit tests, aligning more closely with the practices of human developers and providing a valuable benchmark for the ongoing development of code generation models.