November 2023 Summaries

Deepgram’s example demonstrates how to automatically transcribe and summarize a sales call, then save the generated summary as a Salesforce activity associated with an account and opportunity. Using the Deepgram Python SDK, the workflow sends an audio URL to Deepgram’s prerecorded transcription endpoint with summarization enabled and extracts the short summary from the response. It then uses the simple-salesforce package and Salesforce API credentials—including an email, password, and security token—to find an account by name and retrieve one of its opportunities. The script creates a call event containing the summary, subject, duration, timestamp, and related Salesforce record IDs, allowing users to view the call details in the Activity sections of the relevant account and opportunity. The example can be extended through custom fields, additional metadata, alternative opportunity-selection logic, and customized call details.

Deepgram has released open-source captioning libraries for Python and JavaScript that generate SRT and WebVTT subtitles from speech-to-text transcription data. Designed to work with Deepgram as well as other APIs, the packages use converter classes to transform API-specific JSON responses into a common word-level format containing timestamps and optional punctuation. They support speaker labels when diarization data is available, allow developers to build custom converters for unsupported services, and offer indirect Python support for Whisper through whisper-timestamped or Deepgram-hosted Whisper. The libraries can be installed through pip or npm and include straightforward functions for producing standard caption files, with a Streamlit demonstration showing their use on YouTube videos.

AI’s environmental impact stems from both model training and ongoing inference, with electricity use varying by hardware, model size, and deployment scale. Research cited from universities and industry suggests that training large language models can produce substantial emissions, while end-to-end machine-learning analyses indicate that inference may account for the largest share of power consumption once systems are widely used. Meta reported that training Llama 2 generated 539 tons of carbon-dioxide equivalent emissions, which it says were offset through its sustainability program, illustrating both the scale of individual training runs and the role of corporate mitigation claims. As AI chatbots and other generative systems expand toward search-engine-level usage, their electricity demand could become comparable to that of small countries, increasing both environmental and financial costs. The discussion also highlights efforts by organizations such as Hugging Face to measure AI’s climate effects and suggests that using smaller, reasonably accurate models could reduce compute, emissions, and expense, though the trade-off with performance remains unresolved.

Voice technology and natural language processing have become increasingly accessible beyond engineering and academic settings, making responsible and equitable use more important. The discussion highlights how speech-to-text and related systems often perform best for standardized, high-resource languages such as English, Mandarin, Spanish, and French because training data is more abundant and consistently labeled, while low-resource languages, regional varieties, and marginalized speakers may receive less accurate support. Language standards can enable broad communication and access to tools such as translation, but they may also reflect the speech of socially privileged groups and reinforce existing inequalities. Creating separate standards for underrepresented varieties may improve resources in some cases, yet language boundaries are fluid and the large number of dialects makes comprehensive standardization difficult. Although no simple solution is offered, ongoing research, data-collection efforts, and advocacy are seeking to improve automatic speech recognition for marginalized linguistic communities, people with speech disabilities, and LGBTQ+ users.

Small AI startups face significant challenges in competing with large tech companies due to a lack of resources such as data and computational power. However, they can leverage their agility, innovation, and focused expertise to carve out their niche and stand a chance against bigger corporations. Strategies like niche specialization, collaborations, customer-centric solutions, feature density differentiation, owning new product categories, use of open source and public datasets, innovative monetization models, and focus on talent recruitment and retention can help AI startups to thrive in the competitive landscape.

Deepgram has released Nova-2, a next-generation speech-to-text model that outperforms alternatives in terms of accuracy, speed, and cost. The API now supports multiple languages including Spanish, German, and Hindi, as well as custom model training. Nova-2 boasts an average 30% reduction in word error rate (WER) over competitors for both pre-recorded and real-time transcription, with pricing starting at $0.0043 per minute. The API has been used to transcribe more than 60 million audio minutes across various domains and use cases. Deepgram's commitment to improving spoken language understanding is evident in the continuous enhancement of its speech AI technology.

Whisper-v3, the latest version of OpenAI's automatic speech recognition (ASR) model, has been found to hallucinate more frequently than its predecessor, Whisper-v2, when tested on real-world data. The median Word Error Rate (WER) for Whisper-v3 is 53.4, while Whisper-v2 only has a median WER of 12.7. Users have reported hallucinations in languages like Japanese and Korean as well. The author of this text tested the model on various audio files and found that it performs well with edge cases but struggles with real-world data, leading to high error rates.

The food industry has seen an increase in the use of language AI to improve efficiency and customer experience. During the COVID-19 pandemic, online ordering and delivery services like UberEats and Doordash saw a surge in demand, leading some restaurants and platforms to incorporate voice recognition technology into their systems. Voice ordering is made possible by integrating existing voice assistants or building custom voice recognition systems that use speech recognition, content-based filtering, natural language understanding (NLU), and natural language generation (NLG). Language AI can also be used in drive-thru restaurants and inventory control processes. Challenges include the high cost of implementation, potential user dissatisfaction with voice ordering, and technical issues such as achieving high-speed performances and training speech recognition models for industry-specific vocabulary.

Researchers from University of Florida discovered that AI like ChatGPT can predict next-day stock performance based on financial news headlines, with a strong positive correlation between higher scores from ChatGPT and better returns. NLP has been used in financial analysis since the 1980s, but recent advancements have led to more sophisticated techniques for analyzing real-time user data from social media. Sentiment analysis plays a significant role in current NLP algorithms, with three popular approaches currently being used for portfolio construction and risk modeling: Loughran-McDonald, Google's FinBERT, and Alexandria Technology. Financial forecasting often focuses on stock market and foreign exchange rate prediction due to the complexity of financial data and difficulty obtaining information like corporate financial statements. While AI shows promise in predicting stock market movements, it is still uncertain if these models can be applied to other aspects of the financial sector, such as VC investing in early-stage startups.

Defensibility is crucial for AI startups to grow and thrive, preventing monopolies and promoting innovation. It helps companies differentiate themselves from competitors by building a stand-out product with unique features or better performance in specific areas. A defensible startup has direction and focus on achieving Product-Market Fit (PMF) and identifying its unique selling points. By maintaining defensibility, startups can continue to grow and compete against larger companies in the AI space.

This article highlights five prominent AI researchers who have made significant contributions to the field. Geoffrey Hinton, known as the "godfather of deep learning," developed artificial neural networks and popularized backpropagation algorithms. Fei-Fei Li contributed to computer vision through large-scale datasets like ImageNet and pioneered Natural Scene Understanding. Latanya Sweeney is a leading voice in privacy and data protection, establishing public interest technology as a field. Yoshua Bengio co-founded the Montreal Institute for Learning Algorithms (Mila) and developed word embeddings, LSTMs, and GANs. Jürgen Schmidhuber is credited with developing Long Short-Term Memory networks. These researchers have shaped AI's trajectory and continue to influence its future development.

Deepgram and Amazon Web Services have expanded their partnership by integrating Deepgram's advanced speech-to-text technology with Amazon Connect, a cloud-based contact center service solution. This integration offers enhanced speech recognition, real-time insights, automated call summarization, and workflow automation capabilities. The partnership aims to improve customer experiences, increase operational efficiency, enable data-driven decision making, and provide scalability and flexibility for businesses in various industries.

The article discusses how deep learning can revolutionize the handling of tabular data, a crucial yet often overlooked aspect of artificial intelligence. Despite advancements in language processing and image recognition, modern deep learning methods have not been adequately researched or applied to tabular data. Many research papers and online courses assume that "classical" tree-based models and gradient boosting machines (GBMs) outperform deep learning methods on this type of data. However, the article argues that these studies often use standard benchmark datasets, which are limited in scope and do not represent the entire tabular data domain. Furthermore, the baseline models used for benchmarking in tree-based models are highly optimized state-of-the-art techniques, while neural networks pitted against them tend to be simple MLPs or standard architecture variants. The article highlights that times have changed, and tabular datasets now account for a majority of day-to-day data analysis and processing. The domain space encompassed by these datasets has been greatly expanded due to advancements in data collection techniques. Tree-based models are not necessarily "bad choices" for modern tabular data modeling but may fall short when it comes to large and complicated datasets. Deep learning models have limitations, such as interpretability, lack of data, and inability to preprocess data effectively. However, recent studies from Caglar Aytekin demonstrate that neural networks can be understood like any other decision tree, a massive leap in the field of neural network interpretability. Additionally, tabular GANs have been explored for generating synthetic tabular data, with models such as TGAN and CTGAN providing anonymization features to protect user privacy. Incorporating the best of both worlds, many models have applied the tree-based modeling concept to deep learning models, such as GrowNet, Deep Neural Decision Trees, Neural Oblivious Decision Ensembles, and XBNet. Researchers have also been "stealing" the success of attention-based Transformers for tabular data with models like TabTransformer, TabNet, and SAINT. The article concludes by stating that deep learning models have potential in solving predictive tasks involving tabular data, as evidenced by successful applications such as the Kaggle Mechanism of Action (MoA) competition and the Jane Street Market Prediction. The importance of tabular data modeling cannot be ignored, and with modern techniques, its potential is unbounded.

The article discusses the potential benefits and drawbacks of using Hidden Markov Models (HMMs) in speech recognition applications, comparing them to vintage cars. Despite their proven effectiveness, HMMs are often overlooked in favor of newer, more complex models like RNN-transducers. The author suggests that hybrid machine learning architectures and models, including HMM-based systems, deserve more attention due to their efficiency and practicality. One reason for the lack of enthusiasm towards HMMs is the clunky pipeline associated with them, which can be discouraging for developers. Additionally, there isn't a unified Python library for HMMs like there is for modern machine learning frameworks such as PyTorch or Tensorflow. The author calls for an imperative codebase and a unified codebase to make HMM-based approaches more accessible and attractive to developers. The article concludes by encouraging developers to explore older models, identify gaps, and consider upgrading them if necessary. It emphasizes the importance of not dismissing HMMs outright due to their age or simplicity, as they can still be effective in many applications.

Since its release in late 2022, users have been attempting to "jailbreak" OpenAI's ChatGPT by crafting prompts that trick the model into providing unsafe or controversial responses. This has led to the emergence of prompt engineering as a field and the development of more sophisticated jailbreaking techniques. Researchers have proposed methods for generating universal adversarial prompts, which can be used to consistently elicit harmful or biased responses from language models. The introduction of blackbox jailbreaking presents an even greater challenge, as it does not require access to the target model's architecture or parameters. These developments highlight the need for updated security measures and continuous improvement in AI safety mechanisms.