February 2024 Summaries

Evo, a long-context biological foundation model, has been developed using the StripedHyena architecture, which generalizes across DNA, RNA, and proteins. The model is capable of predicting tasks and generating designs at both molecular and genome scales, with an unprecedented level of sensitivity to single-nucleotide changes. Evo's capabilities include zero-shot gene essentiality testing, zero-shot prediction across DNA, RNA, and protein modalities, CRISPR system generation, and genome-scale generation. The model has been trained on a large corpus of prokaryotic genomic sequences covering 2.7 million whole genomes and is available for use via the Together API and Playground. Evo has the potential to accelerate biological discovery and understanding, as well as be applied to real-world problems such as drug discovery, agriculture, and sustainability. However, further experimental validation is required for the generated sequences.

The pretrain-finetune paradigm has revolutionized machine learning by enabling LLMs to align with distinct user preferences or specialized task requirements through fine-tuning. However, multi-tenant serving is challenging due to expensive storage and serving challenges. Researchers have proposed a novel approach called BitDelta, which decomposes the weights of fine-tuned models into their pre-trained components and an additional delta, allowing for 1-bit quantization without compromising performance. This approach addresses both storage and serving challenges by reducing GPU memory requirements and improving inference speedup. BitDelta is fast, general, and can retain all sorts of fine-tuning information, making it a promising solution for the future of machine learning in multi-tenant settings.