September 2021 Summaries

The 18th issue of The Comet Newsletter highlights recent developments in AI ethics and new technological projects from major tech companies. A report by Reuters investigates how big tech firms like Google and Microsoft are navigating AI ethics, debating self-regulation versus governmental oversight, particularly in areas such as predictive AI and speech technology. The newsletter also introduces a new Python package from TensorFlow, designed to simplify training similarity models using contrastive deep learning, and discusses Google's innovative approach to generating 3D dance movements via a transformer model. Additionally, Facebook AI's Generative Spoken Language Model (GSLM) aims to overcome the limitations of traditional text-based NLP by learning from raw audio signals, enabling advancements in expressive speech generation. These projects underscore ongoing efforts to enhance AI capabilities while addressing ethical considerations and technical challenges.

In issue #18 of The Comet Newsletter, the focus is on recent advancements in AI and machine learning, including a report on how major tech companies are internally addressing AI ethics, with examples from Google and Microsoft, and the debate over whether these companies or regulatory bodies should enforce ethical standards. New projects highlighted include Facebook's Generative Spoken Language Model, which aims to expand NLP capabilities by learning from raw audio without text, Google's innovative 3D dance generation model that uses cross-modal transformers, and a new TensorFlow Python package designed to streamline the training of similarity models through contrastive deep learning. These projects demonstrate diverse approaches to overcoming challenges in AI, from ethics and language model limitations to creative applications like dance generation.

The Comet Newsletter's issue #17 covers a spectrum of topics in machine learning and AI, including emerging skepticism about the correlation between the size of language models and their performance, as exemplified by the FLAN architecture's ability to generalize across tasks. The newsletter features a comprehensive guide on addressing concept drift in production machine learning models, emphasizing the necessity for ongoing adaptation to dynamic real-world conditions and highlighting practical solutions and ethical considerations. It also provides insights into graph neural networks through a two-part series by Google researchers, detailing their structure, functionality, and the implications of graph convolutions. Additionally, the newsletter includes guidance from Stanford instructor Chip Huyen on deploying ML models on the edge, covering aspects of model compatibility, performance, and optimization methods.

The recent Comet ML Office Hours, hosted by The Artists of Data Science, will take a break on September 5th for Labor Day but will resume on September 12th. The session featured discussions on managing machine learning projects, the importance of foundational data infrastructure, and strategies for recharging to combat burnout. Harpreet, the host, emphasized the creation of baseline models and principled experiment management, while data engineer Christian highlighted the challenges of meeting tight deadlines without adequate infrastructure. Additionally, tips for recharging included altering routines and engaging in activities like spending time with children. The discussions often lead to resource sharing, including recommendations for books and materials on machine learning. Participants are encouraged to join these free hour-long sessions held every Sunday and to subscribe to the newly launched Comet Newsletter for insights and updates in the field of data science and ML.

The eighth session of the Comet ML Office Hours, hosted by The Artists of Data Science, covered several engaging topics, including effective management of machine learning (ML) projects, the importance of foundational data infrastructure in convincing business stakeholders, and strategies for recharging to prevent burnout. The discussion, sparked by a query from community member Paul Fentress about job searches post-graduation, emphasized the significance of undertaking projects, with specific advice on creating baseline models and adopting a principled approach to experiment management. Participants also delved into the challenges faced by individual contributors when pressured to meet tight deadlines without adequate infrastructure, offering insights and solutions. Additionally, regular attendee Kris revisited the topic of burnout, prompting the group to share various methods for recharging, such as breaking routines and challenging existing worldviews. The session underscored the value of these virtual gatherings, which take place every Sunday, and highlighted the newly launched Comet Newsletter, promising insights and developments in data science and ML.

The Comet Newsletter's 16th issue covers several intriguing updates in the machine learning landscape, highlighting a forthcoming MLOps webinar by Niko Laskaris and the hiring of AI Ethics Researcher Margaret Mitchell by Hugging Face, a company known for its model hub and open-source tools. Margaret Mitchell's move from Google is seen as a strategic addition to reinforce Hugging Face's ethical framework. The newsletter also delves into the creative potential of AI, with playwright Jennifer Tang's experiment using GPT-3 to write a play, producing mixed but thought-provoking results. Additionally, it features an extractive text summarization project using BERT by Victor Dibia from Cloudera Fast Forward Labs, and discusses a new neural text-to-speech (TTS) approach by Google AI, aimed at improving speech synthesis for people with impairments through the combination of PnG BERT and Non-Attentive Tacotron models.

In issue #16 of The Comet Newsletter, several advancements in machine learning and AI are highlighted, including Hugging Face's hiring of Margaret Mitchell, a former Google AI Ethics Researcher, to enhance its ethical framework. The newsletter discusses a creative endeavor where AI, using GPT-3, attempted to write a play at London's Young Vic theatre, resulting in a mix of outcomes but offering some intriguing "almost human insights." It also covers Victor Dibia's exploration of extractive text summarization, contrasting it with abstractive summarization and redefining it as a sentence classification task. Furthermore, Google AI's development of the PnG NAT model, which combines PnG BERT and Non-Attentive Tacotron for improved text-to-speech synthesis, is presented as a significant step towards enhancing AI-assisted speech fluency and quality.