January 2024 Summaries

Seldon has launched its LLM Module in beta, designed to simplify the deployment and integration of Generative AI (GenAI) into business operations by offering a seamless interface for deploying and serving Large Language Models (LLMs) through local and hosted environments, including OpenAI and Azure services. This module integrates with leading LLM-serving technologies such as vLLM, DeepSpeed, and Hugging Face, providing optimizations that enhance efficiency, reduce latency, and improve resource utilization, making it easier for businesses to deploy sophisticated AI applications like chatbots. The module is part of the broader Seldon ecosystem, which includes model management and monitoring tools, allowing businesses to efficiently manage AI deployments without needing to learn new systems. Seldon emphasizes flexibility by offering a tech-stack agnostic approach, ensuring GenAI applications remain relevant as AI continues to advance and providing enterprises with competitive advantages across various functions like customer service, marketing, and HR. As AI adoption accelerates, particularly among GenZ professionals, Seldon's LLM Module aims to help businesses harness AI's potential to boost productivity, reduce costs, and keep proprietary data secure, while also supporting the development of younger talent.

Seldon, a company founded in 2014 with a mission to accelerate machine learning adoption, has announced a licensing change for its open-source projects. While their primary projects, Core 1, Core 2, and Alibi, have historically operated under the Apache 2.0 license, they will now transition to the Business Source License v1.1 (BSL) as of January 22, 2024. This change will mean that these projects remain free for non-production uses, while production use will require a commercial license. Seldon believes this shift strengthens its commitment to an open core business model, allowing for better commercialization and reflecting the extensive value these projects provide. The MLServer product will remain open source under the Apache 2.0 license. This move is aimed at balancing open-source foundations with commercially licensed enterprise-grade products, aligning with the evolving landscape of MLOps and AI.

Seldon has announced significant updates to its technology, enhancing the Seldon Enterprise Platform and Seldon Core v1 and v2, aimed at improving model deployment, error management, and overall user experience in machine learning operations (MLOps). The Seldon Enterprise Platform now offers improved authentication configurability, support for Model URI in the UI, detailed error messages for model deployment failures, and differentiation between users and machines via OIDC configuration. It also introduces configurable resource allocation for batch job pods and UI support for folder-based batch jobs. Seldon Core v1 and v2 have been updated to include optional user ID requirements for Helm chart management, OAuth 2 SASL mechanism for Confluent Kafka, enhanced error handling, and Kafka configurability. Additionally, Seldon Core v2 supports the deployment of HuggingFace models, offering a new runtime for experimentation and efficiency. The platform has also focused on security and usability improvements to tools like Alibi Detect and Alibi Explain, and important bug fixes and dependency upgrades have been implemented across the system to ensure seamless integration with the latest technologies, delivering a robust framework for scalable and efficient MLOps.