July 2021 Summaries

The Comet Newsletter's issue #11 highlights the introduction of Comet Artifacts, a new suite of tools designed to enhance data management in machine learning (ML) experiment pipelines by allowing teams to log, version, and access data efficiently. These Artifacts create a structured approach to manage experiment outputs and inputs, facilitating the construction of multi-stage pipelines and ensuring managed access to intermediate data. Additionally, the newsletter features industry insights, including a critique of OpenAI Codex's limitations in solving code-based problems compared to other GPT-3 versions, and a discussion on the complexities of automating moral decisions in ML models. It also includes a detailed resource on explaining Transformer models, emphasizing their significance in deep learning through visualizations of their architectures.

Issue #11 of The Comet Newsletter introduces Comet Artifacts, a new tool designed to aid machine learning (ML) teams in logging, versioning, and accessing data throughout their experimentation workflows. This tool allows for the management of data produced in ML experiments by creating versioned objects known as Artifacts, which provide an immutable snapshot of files and assets in a structured format. This innovation supports the construction of multi-stage pipelines and Directed Acyclic Graphs (DAGs), ensuring centralized and versioned data access. The newsletter also highlights industry developments, such as OpenAI Codex, which is built on GPT-3 architecture to assist in AI-driven software development, and explores topics like ML model fairness, transparency, and decision-making ethics. Additionally, it features insights from a detailed resource by Jay Alammar focusing on Transformer architectures, offering various perspectives on this prevalent deep learning model.

Comet Artifacts is a toolset designed to address the complexities of managing machine learning experiment data by providing a structured way to log, version, and browse data across experimentation pipelines. It facilitates the management of outputs such as model predictions and weights, which can serve as inputs for subsequent experiments, by creating versioned objects called Artifacts. These Artifacts are immutable snapshots organized in a logical structure, with metadata, version numbers, tags, and aliases to track their lineage across experiments. This setup enables machine learning teams to construct multi-stage pipelines or Directed Acyclic Graphs (DAGs), ensuring centralized and versioned access to intermediate data. Comet Artifacts simplifies the process of registering and using Artifacts in experiments, requiring minimal lines of code for integration, thereby streamlining data management and enhancing collaboration.

The Industry Q&A series is being relaunched with a virtual event featuring Jakub Jurovych of Deepnote and Abubakar Abid of Gradio, focusing on the evolving importance of collaboration in data science and machine learning development. Scheduled for July 26th at 11 am ET, the event will delve into the paradigm shift towards collaborative tools, the problems they address, and the industry's response to these team-based solutions. Jakub Jurovych, a Cambridge-educated engineer and Forbes 30 Under 30 honoree, leads Deepnote, a data science notebook platform, while Abubakar Abid, an MIT graduate and PhD candidate at Stanford, heads Gradio, which enhances the reliability of machine learning models. The free-to-attend event includes an audience Q&A and is available on-demand for registrants unable to attend live, marking the beginning of a series of similar events.

Comet ML Office Hours, hosted by The Artists of Data Science, will not occur on July 25th but will resume on August 1st due to Harpreet leading a session at the DataScienceGO virtual conference. The recent office hours featured discussions on learning SQL and adjusting to new roles, with Harpreet providing resources such as Azure Data Studio and his Data Science Dream Job course. The session also delved into poor career advice, highlighting issues like controversial advice for attention and rigid claims lacking nuance. The sessions, held every Sunday, offer a platform for engaging discussions on data science and machine learning, and attendees are encouraged to join and register for future events. Additionally, Comet has launched a newsletter to provide insights into data science and machine learning, inviting readers to subscribe for updates.

The team behind RecList is highlighting the growing importance of collaboration in machine learning (ML) by hosting an Industry Q&A series featuring industry leaders Jakub Juryovych of Deepnote and Abubakar Abid of Gradio. Scheduled for July 26th, this virtual event aims to explore the paradigm shift towards collaborative approaches in ML, addressing specific problems these methods can solve and how the industry is adapting to new team-based tools. The discussion promises to delve into the essential nature of collaboration in accelerating ML development, with opportunities for audience interaction and questions. The event underscores a broader trend towards enhancing team workflows in data science and will be available on-demand for those unable to attend live.

The eighth session of Comet ML's Office Hours series, "Seven Simple Steps to Standardizing the Experiment," featured discussions with guests Dr. Doug Blank, Jacques Verre, Dhruv Nair, and Michael Cullan, focusing on both technical and philosophical aspects of data science. The session included practical advice on learning SQL, with resources shared for those starting new roles and needing to quickly acquire new skills, as well as a philosophical discussion about career advice, highlighting the pitfalls of bad guidance and the impacts of imposter syndrome. The session, which is part of a regular series aimed at fostering community engagement in data science and machine learning, will take a brief hiatus due to Harpreet's participation in the DataScienceGO conference but will resume on August 1st. The Comet ML community also recently launched a newsletter offering insights into data science and machine learning developments.

Issue #9 of The Comet Newsletter features a range of topics, including a Mozilla study on YouTube's recommendation system, which is described as a "horrorshow" due to its tendency to promote content that users regret watching, such as misinformation and inappropriate material. The study, based on data from over 37,000 YouTube users, highlights the role of the recommendation algorithm in pushing undesirable content, noting that non-English-speaking users experience more negative outcomes. The newsletter also includes insights from a Turing Lecture by deep learning pioneers discussing the challenges and future directions of AI, particularly the need for AI systems to improve in handling out-of-distribution data and understanding causality. Additionally, it explores the emerging art scene influenced by OpenAI's CLIP model and offers a fictional narrative on the complexities of building a data science team in a mid-stage startup, emphasizing the importance of processes and organizational buy-in for successful data-driven initiatives.

The Comet Newsletter issue #9 explores various topics, including a Mozilla study on YouTube's recommendation system, which highlights the platform's role in promoting controversial content based on crowdsourced data collected from users. The research indicates that 71% of regretful videos were recommended by the algorithm, with non-English-speaking countries reporting worse experiences. The newsletter also features a fictional narrative by Erik Bernhardsson on the challenges of integrating data-driven practices in startups, emphasizing the importance of process over tools. Additionally, it discusses the impact of OpenAI's CLIP model on generative art, and recaps a Turing Lecture by deep learning pioneers, which delves into the advancements and challenges in AI research, particularly in areas like causality and the limitations of current deep learning systems in adapting to real-world variability.

In the latest Comet ML Office Hours, hosted by The Artists of Data Science and streaming on platforms like LinkedIn, YouTube, and Twitch, participants explored the motivations behind pursuing careers in data science and machine learning, emphasizing the importance of these fields as applications for mathematical interest and creativity, and as part of a vibrant community. A significant discussion focused on the balance between showcasing business impacts and core technical strengths in data science resumes, warning against overemphasizing metrics like revenue generation which might detract from presenting a well-rounded project portfolio. Additionally, the session delved into enhancing creativity within the field, with suggestions for habitual creative practice. Numerous resources were recommended, including books and podcasts, to further explore these themes. Regular attendees are encouraged to participate in these free, interactive sessions held every Sunday, and to subscribe to the newly launched Comet Newsletter for more insights into data science and machine learning.

The eighth session of the Comet ML Office Hours series, "Seven Simple Steps to Standardizing the Experiment," featured discussions with Dr. Doug Blank, Jacques Verre, Dhruv Nair, and Michael Cullan, and was hosted by Harpreet Sahota. Streaming live on platforms like LinkedIn, YouTube, and Twitch, the session explored the challenges of extracting business value from data science projects and emphasized the importance of effective communication about the business impacts of DS/ML work. While focusing on metrics like revenue generation is important, the session highlighted the need for a balanced approach in presenting project portfolios, particularly when projects are centered on research and development or suggest long-term strategic changes. Additionally, the session delved into fostering creativity within data science and provided various insights and resources for attendees. The Office Hours, held every Sunday, offer a collaborative space for data scientists to engage, ask questions, and learn from one another, and are complemented by The Comet Newsletter, which provides expert perspectives on data science and machine learning.

The Comet Newsletter's eighth issue explores several significant developments, including GitHub's launch of Copilot, an AI pair programmer built on OpenAI's Codex, which assists developers by generating code from context prompts and has sparked debate over its originality. Additionally, Distill Pub, an interactive journal for Machine Learning publications, has announced an indefinite hiatus due to burnout and evolving perspectives on the role of traditional peer review, suggesting self-publication may be the future of scientific communication. The newsletter also discusses research from the Alignment Forum that uses parameter counts to measure AI progress, noting a significant increase in model complexity since 2016. Lastly, it covers the challenges of using robots in rescue missions, as demonstrated in the aftermath of the Miami condo collapse, highlighting the physical and ethical limitations that prevent effective robotic intervention in such unpredictable environments.

In the eighth session of the Comet ML Office Hours, part of a series called "Seven Simple Steps to Standardizing the Experiment," participants, including Dr. Doug Blank, Jacques Verre, Dhruv Nair, and Michael Cullan, discussed various topics in data science and machine learning. Despite it being a holiday weekend in the U.S., the session featured engaging discussions on an applied machine learning use case in precision agriculture, where Reema Gill introduced an autonomous, sensor-based irrigation system that uses satellite data to optimize irrigation for small-scale farming. The session also offered insights into data science job interviews, differentiating them from software engineering interviews and providing resources for preparation. Additionally, the group delved into technical advice on hyperparameter selection and tuning, emphasizing its importance in building efficient ML models, as demonstrated in a Kaggle competition context. These Office Hours are held weekly and are open to all, with additional content and insights available through The Comet Newsletter.

The Comet Newsletter discusses various topics, including GitHub's Copilot, a new AI tool built on OpenAI's Codex, which helps developers by generating code from context prompts and has generated debate about its originality. Meanwhile, the interactive online journal Distill is taking an indefinite hiatus due to burnout and reconsideration of its role, with the team questioning the value of traditional peer review compared to self-publication. Researchers at the Alignment Forum propose using model parameter counts as a metric for AI progress, emphasizing the increasing complexity of models over time. Lastly, the challenges of using robots in disaster scenarios are highlighted, as exemplified by the limited applicability of robots in the aftermath of the Surfside condominium collapse, due to the unpredictable nature of rubble environments and the current technological limitations.