June 2022 Summaries

Surge AI, in collaboration with Redwood Research, is working on creating advanced adversarial evaluation methodologies for AI models to ensure they align with human values and do not pose existential threats. Their initial project involves developing a classifier with a highly reliable detection rate of violent text, achieved by building an AI "red team" of creative human labelers who devise new strategies to trick the model, thereby improving its robustness. This process involves training labelers to understand the nuances of what constitutes violence and employing creative tactics like logical misdirection and metaphorical language to bypass the model's detection. The insights gained are used to iteratively improve the model, with the ultimate aim of contributing to the broader machine learning community's efforts in tackling safety and alignment challenges. Additionally, Surge AI is exploring the impact of violence filters on text-generation models and is involved in other projects to evaluate the capabilities of frontier models in real-world applications.

The discussion between Scott from Slate Star Codex and Gary Marcus centers on the capabilities and limitations of large language models like GPT-3, particularly in their understanding of intelligence and commonsense reasoning. Marcus critiques these models for lacking true understanding, evidenced by examples where GPT-3 fails to exhibit logical reasoning, such as suggesting a lawyer might wear a bathing suit to court. Scott counters by arguing that what Marcus sees as failures might actually reflect the model's attempt to mimic human creativity or humor. The debate extends to how humans would perform on similar tasks, with varying outcomes that sometimes align with GPT-3's responses, suggesting that the evaluation of AI's intelligence can be subjective and context-dependent. This exploration raises broader questions about the criteria for judging AI's capabilities and the potential for AI models to demonstrate qualities akin to human creativity.

Surge AI collaborated with OpenAI to create the GSM8K dataset, consisting of 8,500 grade school math problems designed to enhance the problem-solving capabilities of language models like GPT-3. The project involved developing diverse math problems with clear solutions to train AI models in natural language processing and reasoning. The dataset creation emphasized the importance of high-quality data labeling by utilizing a team of mathematically proficient individuals, ensuring problem diversity and mathematical correctness. The dataset is not only used by OpenAI but has also been adopted by other research labs, including Google, for their advanced AI models. Additionally, the dataset's development process highlighted the often-overlooked importance of dataset inspection and quality, as inaccuracies can undermine data-driven projects.