November 2023 Summaries

Small AI startups face significant challenges in competing with large tech companies due to a lack of resources such as data and computational power. However, they can leverage their agility, innovation, and focused expertise to carve out their niche and stand a chance against bigger corporations. Strategies like niche specialization, collaborations, customer-centric solutions, feature density differentiation, owning new product categories, use of open source and public datasets, innovative monetization models, and focus on talent recruitment and retention can help AI startups to thrive in the competitive landscape.

Deepgram has released Nova-2, a next-generation speech-to-text model that outperforms alternatives in terms of accuracy, speed, and cost. The API now supports multiple languages including Spanish, German, and Hindi, as well as custom model training. Nova-2 boasts an average 30% reduction in word error rate (WER) over competitors for both pre-recorded and real-time transcription, with pricing starting at $0.0043 per minute. The API has been used to transcribe more than 60 million audio minutes across various domains and use cases. Deepgram's commitment to improving spoken language understanding is evident in the continuous enhancement of its speech AI technology.

Whisper-v3, the latest version of OpenAI's automatic speech recognition (ASR) model, has been found to hallucinate more frequently than its predecessor, Whisper-v2, when tested on real-world data. The median Word Error Rate (WER) for Whisper-v3 is 53.4, while Whisper-v2 only has a median WER of 12.7. Users have reported hallucinations in languages like Japanese and Korean as well. The author of this text tested the model on various audio files and found that it performs well with edge cases but struggles with real-world data, leading to high error rates.

The food industry has seen an increase in the use of language AI to improve efficiency and customer experience. During the COVID-19 pandemic, online ordering and delivery services like UberEats and Doordash saw a surge in demand, leading some restaurants and platforms to incorporate voice recognition technology into their systems. Voice ordering is made possible by integrating existing voice assistants or building custom voice recognition systems that use speech recognition, content-based filtering, natural language understanding (NLU), and natural language generation (NLG). Language AI can also be used in drive-thru restaurants and inventory control processes. Challenges include the high cost of implementation, potential user dissatisfaction with voice ordering, and technical issues such as achieving high-speed performances and training speech recognition models for industry-specific vocabulary.

Researchers from University of Florida discovered that AI like ChatGPT can predict next-day stock performance based on financial news headlines, with a strong positive correlation between higher scores from ChatGPT and better returns. NLP has been used in financial analysis since the 1980s, but recent advancements have led to more sophisticated techniques for analyzing real-time user data from social media. Sentiment analysis plays a significant role in current NLP algorithms, with three popular approaches currently being used for portfolio construction and risk modeling: Loughran-McDonald, Google's FinBERT, and Alexandria Technology. Financial forecasting often focuses on stock market and foreign exchange rate prediction due to the complexity of financial data and difficulty obtaining information like corporate financial statements. While AI shows promise in predicting stock market movements, it is still uncertain if these models can be applied to other aspects of the financial sector, such as VC investing in early-stage startups.

Defensibility is crucial for AI startups to grow and thrive, preventing monopolies and promoting innovation. It helps companies differentiate themselves from competitors by building a stand-out product with unique features or better performance in specific areas. A defensible startup has direction and focus on achieving Product-Market Fit (PMF) and identifying its unique selling points. By maintaining defensibility, startups can continue to grow and compete against larger companies in the AI space.

This article highlights five prominent AI researchers who have made significant contributions to the field. Geoffrey Hinton, known as the "godfather of deep learning," developed artificial neural networks and popularized backpropagation algorithms. Fei-Fei Li contributed to computer vision through large-scale datasets like ImageNet and pioneered Natural Scene Understanding. Latanya Sweeney is a leading voice in privacy and data protection, establishing public interest technology as a field. Yoshua Bengio co-founded the Montreal Institute for Learning Algorithms (Mila) and developed word embeddings, LSTMs, and GANs. Jürgen Schmidhuber is credited with developing Long Short-Term Memory networks. These researchers have shaped AI's trajectory and continue to influence its future development.

Deepgram and Amazon Web Services have expanded their partnership by integrating Deepgram's advanced speech-to-text technology with Amazon Connect, a cloud-based contact center service solution. This integration offers enhanced speech recognition, real-time insights, automated call summarization, and workflow automation capabilities. The partnership aims to improve customer experiences, increase operational efficiency, enable data-driven decision making, and provide scalability and flexibility for businesses in various industries.

The article discusses how deep learning can revolutionize the handling of tabular data, a crucial yet often overlooked aspect of artificial intelligence. Despite advancements in language processing and image recognition, modern deep learning methods have not been adequately researched or applied to tabular data. Many research papers and online courses assume that "classical" tree-based models and gradient boosting machines (GBMs) outperform deep learning methods on this type of data. However, the article argues that these studies often use standard benchmark datasets, which are limited in scope and do not represent the entire tabular data domain. Furthermore, the baseline models used for benchmarking in tree-based models are highly optimized state-of-the-art techniques, while neural networks pitted against them tend to be simple MLPs or standard architecture variants. The article highlights that times have changed, and tabular datasets now account for a majority of day-to-day data analysis and processing. The domain space encompassed by these datasets has been greatly expanded due to advancements in data collection techniques. Tree-based models are not necessarily "bad choices" for modern tabular data modeling but may fall short when it comes to large and complicated datasets. Deep learning models have limitations, such as interpretability, lack of data, and inability to preprocess data effectively. However, recent studies from Caglar Aytekin demonstrate that neural networks can be understood like any other decision tree, a massive leap in the field of neural network interpretability. Additionally, tabular GANs have been explored for generating synthetic tabular data, with models such as TGAN and CTGAN providing anonymization features to protect user privacy. Incorporating the best of both worlds, many models have applied the tree-based modeling concept to deep learning models, such as GrowNet, Deep Neural Decision Trees, Neural Oblivious Decision Ensembles, and XBNet. Researchers have also been "stealing" the success of attention-based Transformers for tabular data with models like TabTransformer, TabNet, and SAINT. The article concludes by stating that deep learning models have potential in solving predictive tasks involving tabular data, as evidenced by successful applications such as the Kaggle Mechanism of Action (MoA) competition and the Jane Street Market Prediction. The importance of tabular data modeling cannot be ignored, and with modern techniques, its potential is unbounded.

The article discusses the potential benefits and drawbacks of using Hidden Markov Models (HMMs) in speech recognition applications, comparing them to vintage cars. Despite their proven effectiveness, HMMs are often overlooked in favor of newer, more complex models like RNN-transducers. The author suggests that hybrid machine learning architectures and models, including HMM-based systems, deserve more attention due to their efficiency and practicality. One reason for the lack of enthusiasm towards HMMs is the clunky pipeline associated with them, which can be discouraging for developers. Additionally, there isn't a unified Python library for HMMs like there is for modern machine learning frameworks such as PyTorch or Tensorflow. The author calls for an imperative codebase and a unified codebase to make HMM-based approaches more accessible and attractive to developers. The article concludes by encouraging developers to explore older models, identify gaps, and consider upgrading them if necessary. It emphasizes the importance of not dismissing HMMs outright due to their age or simplicity, as they can still be effective in many applications.

Since its release in late 2022, users have been attempting to "jailbreak" OpenAI's ChatGPT by crafting prompts that trick the model into providing unsafe or controversial responses. This has led to the emergence of prompt engineering as a field and the development of more sophisticated jailbreaking techniques. Researchers have proposed methods for generating universal adversarial prompts, which can be used to consistently elicit harmful or biased responses from language models. The introduction of blackbox jailbreaking presents an even greater challenge, as it does not require access to the target model's architecture or parameters. These developments highlight the need for updated security measures and continuous improvement in AI safety mechanisms.