Fixing All Gemma Bugs
Blog post from Unsloth
Unsloth has dedicated the past week to addressing and resolving various bugs in the Gemma model, initially highlighted by Google. These bugs included issues with loss value discrepancies, precision problems with embedding calculations, and improper casting in Keras, particularly affecting the RoPE and GELU functions. Efforts have been made to correct these through updates to Colab notebooks and ongoing pull requests, such as the approximate GELU and RoPE fixes, which are now partially resolved in version 4.38.2 of the Hugging Face transformers. Despite being a small team without external funding, Unsloth continues to share progress and invites support through donations and community engagement on platforms like Discord and Twitter.
| Trend | Post Mentions | Total Month Mentions | Posts | Companies | MoM |
|---|---|---|---|---|---|
| TPUs | 2 | 7 | 6 | 4 | 0% |
| Vector Search | 2 | 1,815 | 230 | 71 | -13% |
| AI Model Fine-tuning | 1 | 434 | 113 | 72 | -8% |
Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.