November 2025 Summaries

Cohere and SAP have expanded their partnership to make Cohere's agentic AI platform, North, available on SAP’s Cloud infrastructure and Business Technology Platform, aiming to provide sovereign and secure AI solutions across Europe and beyond. This collaboration builds on their initial partnership, which integrated Cohere’s enterprise-grade generative and search models into the SAP Business Suite, benefiting industries such as finance, healthcare, manufacturing, and the public sector. The partnership focuses on aligning with data sovereignty requirements, enabling customers to deploy AI solutions that are compliant with local governance while ensuring data security and operational independence. By integrating Cohere’s AI capabilities into SAP’s ecosystem, customers can enhance efficiency without the complexities of AI engineering, thus supporting enterprise innovation without compromising compliance. Both SAP's Chief Technology Officer, Philipp Herzig, and Cohere's Co-Founder and CEO, Aidan Gomez, emphasize the importance of AI sovereignty in the European market and the potential for improved efficiency through this strategic alliance.

Cohere provides a HIPAA-compliant Business Associate Agreement (BAA) to support healthcare providers, insurers, and technology partners in developing secure, custom models that align with specific business needs while adhering to federal standards for protecting personal health information (PHI). By working with Cohere, entities can establish a BAA through discussions with an account executive, ensuring compliance checks and commercial agreements are met for custom model development, which is then deployed in a private environment. The company emphasizes robust security measures, including encryption and compliance monitoring, to protect PHI. While Cohere offers private deployments suitable for HIPAA-regulated entities without requiring a BAA, managed cloud AI platforms hosted by third-party providers necessitate a BAA with the respective provider. Cohere-hosted products and applications are not currently suitable for HIPAA-regulated entities as they do not offer a BAA for these services, underscoring Cohere's commitment to data security and privacy in delivering innovative solutions to healthcare organizations.

Cohere has introduced a high-performance caching mechanism called Shared Memory IPC Caching to the vLLM project, which significantly reduces the data transfer overhead in multi-process LLM inference systems by keeping large multimodal inputs in shared memory. This approach bypasses redundant inter-process communication (IPC) and enables faster, more efficient inference at scale, particularly with large inputs and multiple concurrent GPU workers. Traditional IPC methods often encounter performance bottlenecks due to repeated large data transfers, especially with multimodal inputs like images or audio. Shared Memory IPC Caching overcomes these limitations by allowing a single shared cache to be accessed directly by both sender and receiver processes, eliminating ordering assumptions and redundant data copies. This advancement promises substantial improvements in prefill throughput and time to first token (TTFT) metrics, as demonstrated by benchmarks showing significant performance gains. The new mechanism is versatile and can be applied beyond LLM inference to enhance performance in any application where IPC caching helps reduce redundant data transfers.