May 2023 Summaries

Poet Vicuna-13B, an iteration of Vicuna-13B designed to generate poetry with specified syllabic patterns, represents an effort to enhance open-source large language models for creative tasks. This project, part of an early-stage initiative, emphasizes the complexity of generating poetic structures such as syllable counts and rhyme schemes, which even advanced models like GPT-4 struggle with due to their lack of inherent understanding of language nuances like syllables. Traditional methods like prompt engineering and training have limitations, prompting exploration of alternative approaches such as the introduction of bragi, a library that applies line-level syllabic constraints using a logit warper to manage the generative process. Inspired by previous research, this method dynamically adjusts token selection to maintain desired metric structures, allowing for creative output that adheres to specific patterns. While Poet Vicuna-13B still faces challenges like incomplete adherence to syllabic patterns, the project highlights the potential for combining rule-based interventions with curated training data to create sophisticated poetry generators, inviting community involvement to further explore and develop these innovations.

Replicate experienced a significant outage on May 11, 2023, affecting both its website and API, primarily due to issues with its PostgreSQL database, which is central to its platform. The outage was triggered by a newly implemented asynchronous update feature that inadvertently caused simultaneous INSERT queries for the same prediction ID, leading to increased query latencies and eventual exhaustion of the database connection pool. Despite initial assumptions that the previous day's database resizing might have been the cause, the real issue stemmed from the asynchronous updates creating a dangerous query pattern. As a result, the replicate.com website faced intermittent failures, although customers using webhooks to receive updates were less affected. The company responded by disabling the problematic feature, which restored normal operations, and is now focused on redesigning the feature to avoid similar issues in the future. This experience has provided valuable insights into the system's constraints, prompting a review of potential database hotspots and a determination to improve resilience against database disruptions.