Anyscale Endpoints: JSON Mode and Function calling Features

The Anyscale platform has introduced JSON mode and function calling capabilities on its LLM API offering, allowing users to tailor outputs to specific schema requirements and enable models to use APIs effectively. The Mistral-7B model is the first to support these features in preview, with plans to extend them to additional models soon. Function calling enables models to select the right function and arguments for a given task, while JSON mode ensures that outputs adhere to valid JSON formats. A test dataset has been created to evaluate the performance of this feature across multiple models, including Mistral-7B, which performs similarly to gpt-3.5-turbo in single-turn interactions. The platform is now available as part of the Anyscale Platform, with users able to try out these features with Anyscale Endpoints today.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.