Fine-tuning Gemma 2 with Keras - and an update from Hugging Face

The Gemma 2 model, a recent release in the Keras framework, is available in two sizes, 9 billion and 27 billion parameters, with both standard and instruction-tuned variants, and is built on the combination of Keras and JAX to handle these large models. It supports distributed fine-tuning on TPUs/GPUs using model parallelism, allowing its substantial weights to be partitioned across multiple accelerators, facilitated by JAX’s XLA compiler. The framework introduces the keras.distribution.ModelParallel API, enabling users to specify how model weights are sharded layer by layer in a streamlined manner. The integration with Hugging Face expands access, enabling users to load fine-tuned weights for supported models in KerasNLP, with plans for compatibility with other Transformers models. Additionally, the PaliGemma model, leveraging the Gemma language model and the SigLIP vision model, offers robust performance across various vision-language tasks and is available through multiple platforms.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.