New in April 2024

In April 2024, Baseten released several best-in-class large language models (LLMs) in different sizes, ranging from 3.8 billion to 141 billion parameters, offering flexibility for trade-offs between cost and output quality. These models can be deployed with optimized techniques such as TensorRT-LLM implementations, FP8 quantization, and continuous batching for improved inference efficiency. Additionally, Baseten introduced streaming endpoints for its LLMs, enabling real-time text-to-speech synthesis capabilities. The company also emphasized the importance of CI/CD pipelines for AI models, providing a model management API to build customized tooling for deployment, and introducing a new feature that allows for more reliable deployment status tracking. These updates expand the possibilities for building with AI and improve the stability of production deployments.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.