YC-Bench: Can Your AI Agent Run a Startup Without Going Bankrupt?

YC-Bench is a benchmark designed to evaluate the performance of large language models (LLMs) by simulating the management of a startup over the course of a year, encompassing tasks such as hiring decisions, dealing with challenging clients, and meeting tight deadlines. Out of 12 advanced models tested, only three managed to turn a profit while the rest faced bankruptcy, offering insights into the capabilities and limitations of LLMs in handling complex, long-term business operations. The creators encourage users to engage with the YC-Bench repository and Collinear's SimLab for further exploration and improvement of AI agents in long-horizon tasks.