Use any* LLM with @mux/ai

Mux has expanded its open-source @mux/ai video intelligence toolkit beyond hosted providers such as OpenAI, Anthropic, and Google by adding first-class Baseten support and a general OpenAI-compatible provider for endpoints including Ollama, vLLM, SGLang, Together AI, Fireworks, and OpenRouter. Developers can configure these providers with an endpoint URL, API key, and explicit model name, allowing local, self-hosted, or fine-tuned open-weight models to run existing workflows for video summaries, tagging, moderation questions, caption analysis, chapters, translation, and editing. Video-oriented workflows require vision-capable models and JSON Schema structured-output support, although base64 image submission can accommodate endpoints unable to retrieve remote storyboard URLs, while text-only workflows have fewer requirements. Mux provides verified-model notes and a validation script but does not evaluate user-hosted models or estimate their costs, leaving quality, latency, and infrastructure expenses to users. The update also improves retry handling for occasional malformed or truncated open-model responses, while preserving non-retry behavior for content-policy refusals, and it enables deployments such as LoRA-fine-tuned Mistral models to integrate directly without maintaining a fork.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.