Comparing Progressive Discovery and Semantic Search for Powering Dynamic MCP

Chase Crumbaugh's article discusses a significant update to the Gram OSS repository, which introduces dynamic toolsets to manage token usage efficiently on MCP servers. Traditional static toolsets load all tools into an AI agent's context window at once, leading to excessive token consumption, particularly problematic as toolsets grow in size. To address this, two dynamic approaches, progressive and semantic search, are proposed. Progressive search uses hierarchical meta-tools to discover and execute necessary tools, while semantic search employs embeddings for faster, natural language-driven tool discovery. Both methods dramatically reduce token use and maintain consistent performance across varying toolset sizes. Initial tests show that dynamic toolsets can handle complex tasks more efficiently than static ones, offering scalable, cost-effective solutions for large APIs. The implementation involves exposing meta-tools for discovery and deferring detailed tool schema loading until explicitly required, enhancing efficiency. While still experimental, dynamic toolsets promise better scalability and predictable costs, making them valuable for large API operations without breaching context window limits.