At Sequoia’s AI Ascent conference, the discussion focused on the limitations of agents, particularly in planning, UX, and memory. The post dives into the challenges of planning and reasoning for agents built with large language models (LLMs), highlighting the difficulty in executing long-term plans due to the limitations of current LLM capabilities, such as context window constraints. Developers often use function calling to guide LLM actions, a feature that OpenAI introduced in 2023, allowing for more structured task execution. However, improvements in planning are still necessary, and these often involve providing the LLM with comprehensive information and employing cognitive architectures to manage reasoning and task execution. These architectures can be general-purpose or domain-specific, with the latter often proving more effective for specific applications, as evidenced by the AlphaCodium project's success with its "flow engineering" methodology. The future of planning and reasoning in AI is expected to see advancements in model intelligence, but the need for clear communication of tasks to LLMs will persist, ensuring that prompt engineering and custom architectures remain vital. Tools like LangGraph are being developed to enhance the controllability and reliability of these cognitive architectures, preparing them for the evolution of AI capabilities.