June 2024 Summaries

Google has expanded developer access to its Gemini AI models, enabling innovations such as a 2 million token context window in Gemini 1.5 Pro and code execution capabilities to enhance accuracy in data reasoning tasks. This includes the introduction of context caching to reduce costs and support for code execution in a sandbox environment with Python code. The new Gemma 2 model is available in Google AI Studio for experimentation, aiming to make AI accessible to all developers. The Gemini 1.5 Flash model addresses speed and affordability, already being used in applications like Envision for real-time environmental descriptions for visually impaired users, Plural for policy analysis, Zapier for video automation, and Dot for personalized AI tasks. Google is also rolling out text tuning for Gemini 1.5 Flash, enhancing production robustness and reliability, with plans to offer these features to all developers by mid-July via the Gemini API and Google AI Studio.

The Gemma 2 model, a recent release in the Keras framework, is available in two sizes, 9 billion and 27 billion parameters, with both standard and instruction-tuned variants, and is built on the combination of Keras and JAX to handle these large models. It supports distributed fine-tuning on TPUs/GPUs using model parallelism, allowing its substantial weights to be partitioned across multiple accelerators, facilitated by JAX’s XLA compiler. The framework introduces the keras.distribution.ModelParallel API, enabling users to specify how model weights are sharded layer by layer in a streamlined manner. The integration with Hugging Face expands access, enabling users to load fine-tuned weights for supported models in KerasNLP, with plans for compatibility with other Transformers models. Additionally, the PaliGemma model, leveraging the Gemma language model and the SigLIP vision model, offers robust performance across various vision-language tasks and is available through multiple platforms.

At the I/O Connect event in Berlin, Google showcased advancements in AI technology and developer tools to an audience of over 1,300 developers from across Europe, the Middle East, and Africa. Key announcements included the release of Gemma 2, featuring enhanced performance and safety, and updates to the Gemini API that facilitate code execution and context caching. Google emphasized its commitment to making AI accessible, introducing initiatives such as the Gemini API Developer Competition and new features for mobile and web development, including Gemini integration in Android Studio and Gemini Nano in Chrome's Prompt API. Additionally, Google announced new cloud regions and expanded its sovereign cloud solutions to more countries, supporting developers with data residency and compliance needs. The event also highlighted Google's support for startups through accelerator programs, with a focus on sustainability and AI-driven innovation. The next I/O Connect event is scheduled in Bengaluru, promising further developments in Google's developer ecosystem.

Model Explorer is a newly released graph visualization tool from Google AI Edge, designed to help developers visualize and understand the architecture and data flow of large machine learning models, especially when optimizing for edge devices. Originally developed for Google's internal use, this tool supports popular ML frameworks such as JAX, PyTorch, TensorFlow, and TensorFlow Lite, and offers features like hierarchical visualization, customizable data overlays, and a smooth 60 FPS rendering experience using WebGL and three.js. Model Explorer can run locally, in Colab notebooks, or within Python files, allowing users to visualize models from multiple sources and formats, including through specialized APIs for PyTorch models. It simplifies tasks such as debugging model conversion errors, analyzing performance and numerical accuracy, and providing insights into model architecture by breaking down complex models into hierarchical layers. It has been effectively utilized by teams at Google, including Waymo and Google Silicon, to optimize on-device models, and future enhancements are planned to refine UI features and improve tool extensibility.

Google's recent announcements at the I/O event introduced new products and tools, including the Gemini API, to empower developers to create innovative AI-driven applications. Highlighting the integration of these tools, Google developed the I/O Crossword, a game designed to showcase the capabilities of Flutter, Firebase, and the Gemini API. The crossword puzzle, which features technology-related words sourced from Google I/O keynote speeches, allows players to interact collaboratively while utilizing a "Hint" feature powered by the Gemini API for assistance. The game employs Flutter for its user interface and performance across platforms, Firebase for backend functionality and real-time data handling, and Cloud Run for efficient backend management. This initiative serves as both a demonstration of Google's technologies and an invitation for developers to participate in the Gemini API Dev Competition, with resources provided to facilitate the development process.

Google Developer Student Clubs (GDSC) are celebrating the fifth year of their Solution Challenge, where university students globally are addressing the United Nations' 17 Sustainable Development Goals using Google technologies such as AI, Android, Google Cloud, and Firebase. The competition's Demo Day, scheduled for June 27th, will be livestreamed on YouTube, featuring the top 10 teams selected from 100 global projects. These teams will present their innovative solutions to a panel of Google judges and a worldwide audience, aiming to tackle issues like poverty, health, education, and sustainability through their projects. Examples of these projects include a mobile app for early eye disease detection in Sub-Saharan Africa, an app with a voice assistant for dementia patients, an AI-powered nutritional assistant, and a platform for redistributing surplus food to combat hunger. Participants and viewers can engage by voting for a People's Choice Award and sharing their thoughts online using #GDSCDemoDay.

Best Phones Forever: AI Roadtrip is an innovative campaign by Google that uses generative AI to engage fans by allowing them to suggest road trip locations for the characters Pixel and iPhone, thereby creating custom video responses. Through the use of a tool developed in collaboration with The Mill and Left Field Labs, the campaign employs Google AI models such as Gemini for script generation, Imagen for background images, and Cloud Text-to-Speech for audio. This setup enables the production of fan-driven content at scale, with human creatives curating and enhancing the AI-generated materials to maintain the campaign's tone and humor. By using AI to generate scripts, backgrounds, and audio, the initiative aims to produce hundreds of unique mini-episodes in a single day, showcasing the potential of AI in creative content development and offering a glimpse into future applications of these technologies.

Labs.google introduces GenType, an innovative tool that utilizes Google's Imagen model to transform a single user input into an entire alphabet using generative AI. By specifying a material or object, such as "grape jelly, on toast, aerial shot," users can create a visually unique alphabet where each letter is crafted from the chosen theme. The tool automates the process, sending out 26 requests for each letter, and allows for customization, including the ability to type phrases, save or copy letters, and regenerate specific letters for different styles. GenType emphasizes the importance of specificity in prompts by focusing on foreground, background, and style to maintain a consistent aesthetic. Designed to be flexible and creative, GenType is already being used for various projects, from jewelry design to event invites, showcasing its potential to make AI more approachable and fun. Google encourages further experimentation with generative AI by inviting submissions of innovative projects to its platform.

The Google Developer Program is a new initiative offering developers worldwide access to a variety of tools, benefits, and communities to enhance their skills and projects. By joining, developers can receive AI assistance through chat interfaces on Google platforms, explore Google APIs without coding, and gain insights into Google tools and documentation. The program also includes Project IDX workspaces for efficient app development and offers free cloud learning credits through the Google Cloud Innovators program, which provides interactive labs and skill development resources. Joining Google-sponsored developer communities allows members to connect with peers and experts, access exclusive events, and update their profiles to tailor their learning journey, track progress, and showcase achievements. The program promises additional tools and resources in the future to further aid developers in their learning and development journey.