Integrate Local and Open-Source Core AI Models Into Stream Chat

WWDC 26 introduced Apple's Core AI framework, enabling developers to integrate third-party AI models into their Swift/SwiftUI projects, thus expanding beyond Apple's traditional Foundation Models and Core ML. This new framework, still in beta, requires macOS Golden Gate and specific system updates to function, allowing models from platforms like Hugging Face to be converted into Apple's .aimodel format for on-device execution. A demonstration of this capability is provided through a SwiftUI chat app using Stream's Chat SDK, which incorporates AI features such as real-time chat, voice input with speech-to-text conversion, and AI text refinement, all running locally on iOS devices. Additionally, the app enables photo Q&A functionality by leveraging vision-language models, allowing users to interact with image attachments in chat. Core AI enhances app flexibility by determining optimal processing on CPU, GPU, or Apple's Neural Engine, and while primarily available for iOS, similar experiences can be adapted for other platforms using Stream's SDKs. The open-source nature of Core AI encourages exploration and customization, with source code and models available on GitHub for developers to experiment and innovate further.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.