May 2023 Summaries

Vickie Li, a Senior Security Engineer at Instacart and a Web Google Developer Expert (GDE), is celebrated during AAPI Heritage Month for her contributions to the tech industry and her advocacy for diversity. Initially pursuing a career in development, she discovered a passion for security, leading her to participate in bug bounties and eventually blog about her experiences. Vickie emphasizes the importance of representation and community support within the tech industry, particularly for the AAPI community, as it inspires future generations and fosters a sense of belonging. Her attendance at events like Women Techmakers at I/O ‘23 highlighted the solidarity and empowerment that comes from connecting with peers of similar backgrounds. Vickie encourages AAPI developers to seek relatable mentors who understand cultural nuances, as this guidance can be instrumental in overcoming industry challenges. Through the GDE program, she has witnessed the success of individuals with similar experiences, reinforcing her belief in the power of diversity and mentorship in tech.

During Google I/O 2023, Google Pay announced several new features aimed at enhancing payment security and integration simplicity. These updates include expanding zero fraud liability protection to eligible Visa transactions, improving card authentication through OTP codes or banking apps, and introducing Secure Payment Authentication to reduce fraud while maintaining user convenience. A redesigned Google Pay button, based on Material 3 design principles, offers customization options and is currently available in beta. Additionally, an upgraded test suite with PSP test cards enhances end-to-end testing capabilities, and virtual cards have been widely adopted to facilitate secure online transactions. Autofill is receiving visual updates and expanded functionality, including bank account payment flows in Europe, to streamline checkout experiences. Overall, Google Pay is focused on increasing the number of verified payment forms and improving the security and ease of online payments across its ecosystem.

Google's coding competitions, which began with Code Jam in 2003, have concluded after 20 years, marking the end of an era that saw millions of participants from around the world engage in algorithmic problem-solving challenges. These competitions, which expanded to include Kick Start and Hash Code, provided a platform for programmers of all skill levels to test their abilities and develop solutions capable of scaling on multiple machines. The final event, held on April 15, 2023, featured simultaneous online rounds of varying difficulty, allowing participants to compete one last time. As a tribute to the competitions' legacy, the problems and their solutions have been archived in a GitHub repository called coding-competitions-archive, where they remain accessible for practice and reference. The community's journey over the years has been celebrated for its collaborative spirit, creativity, and the shared enjoyment of coding, leaving a lasting impact on those who participated.

Erick Wendel, a Web Google Developer Expert from Brazil, leveraged his passion for open-source projects to resolve a longstanding bug in Node.js, a widely-used JavaScript runtime. The issue involved the child process module, which manages background tasks, and was affecting major libraries like Facebook Jest and Yarn. Erick discovered a temporary solution using a setTimeout function, sparking further investigation and collaboration with the open-source community. Working with notable contributors like Anna Henningsen, Erick contributed to a seamless fix by implementing a method to queue messages until the child process is ready, preserving existing applications' functionality. His efforts not only resolved the issue but also elevated him to a core member of the Node.js project, allowing him to review pull requests and participate in community discussions. Erick's journey underscores the power of open-source collaboration in advancing technology and personal growth, leading him to share his experiences at global conferences and through educational content on platforms like YouTube.

Google's Partner Innovation team is developing Generative AI templates to explore the integration of large language models (LLMs) with Google APIs, starting with a travel industry demo. This open-source project leverages the PaLM API and Google Maps Places API to create personalized travel itineraries through conversational interactions. The demo highlights the potential of LLMs in simplifying complex tasks like travel planning by integrating information from various sources, although it acknowledges the current limitations of LLMs in terms of accuracy and reliability. The project allows for seamless third-party integrations, enabling users to book travel arrangements directly through the chat interface. Google collaborates with partners such as Agoda to enhance user experiences and create business value by offering personalized, conversational travel planning that can adapt to user preferences, weather conditions, and other variables. The technology's application extends beyond travel, potentially impacting industries like retail, fashion, media, and automotive by offering scalable personalization and discovery experiences.

Google has launched a new Google Pay button view for Android, incorporating the latest Material 3 design principles with both dark and light themes, and enhanced customization options to align with users' checkout experiences. This new feature is also available to Jetpack Compose developers through an open-source library called compose-pay-button, which simplifies the integration of the Google Pay button into Android applications using Kotlin. The button supports various labels and allows for customization via a Jetpack Compose modifier. A similar library, compose-wallet-button, has been released for Google Wallet, which includes both regular and condensed button versions, offering a consistent API and assets as found on the Google Wallet developer site. Developers are encouraged to explore the GitHub repositories for detailed information and integration instructions.

In an interview celebrating Global Accessibility Awareness Day, Martine Dowden, a Web Google Developer Expert (GDE) and CTO of Andromeda Galactic Solutions, emphasizes the vital role of accessibility in web design, which ensures websites are usable by everyone, including the 16% of the population with disabilities. Dowden, co-author of "Approachable Accessibility: Planning for Success," advocates for integrating accessibility principles from the early stages of design, similar to security measures, noting the limitations of relying solely on automated tools like Lighthouse for accessibility checks. She highlights the importance of manual reviews and references the Web Content Accessibility Guidelines (WCAG) as a crucial resource for developers while recommending Angular for its built-in accessibility notes. With 71% of users with disabilities abandoning sites due to inaccessibility, Dowden encourages designers and developers to gradually acquire accessibility skills, akin to learning any new skill, to ensure inclusive web experiences.

Google's Partner Innovation team has developed a customizable AI-powered character template to demonstrate the potential of Large Language Models (LLMs) combined with Google APIs to create interactive 3D avatars for various industry applications. This template allows developers to customize avatars' personalities, backstories, and knowledge bases, enabling them to serve as specialized experts in text or verbal interactions. A notable example is Buddy, a friendly dog avatar with a detailed backstory, which illustrates the ability to alter character roles to fit different contexts, such as an insurance agent. The technical implementation involves using Google's speech-to-text and text-to-speech services, along with the MediaPipe framework, to create a natural and engaging conversational experience through animated avatars. This technology has diverse applications, including gaming, education, tech support, and eCommerce, where it can enhance user experiences by bringing characters and brand assets to life. The project, open-sourced for developers, exemplifies the fusion of advanced language models and avatars to expand possibilities in environments like the metaverse, as demonstrated in collaboration with platforms like ZEPETO.

Google's Partner Innovation team is developing Generative AI Templates that integrate Large Language Models (LLMs) with Google APIs to address industry-specific use cases, exemplified by their MoodFood project. MoodFood is a whimsical, AI-driven recipe generator that acts as a 'Food Therapist', creating unique and humorous recipes based on users' moods and situations. It uses the PaLM API to creatively generate, format, and safeguard content, while also allowing integration with other services for enhanced user experiences like direct ingredient purchase. The project demonstrates the potential of LLMs in innovative product discovery across various sectors, such as music and travel, and highlights Google's collaboration with partners like Swiggy to revolutionize customer interaction through AI. The project is open-sourced, inviting developers to build upon these foundations, with further developments supported by global collaborations.

PaLM 2, Google's advanced generative AI model, is now available to developers through the PaLM API and MakerSuite, facilitating the creation of AI applications such as content generation, dialog agents, and code generation. The model, fine-tuned for improved reasoning and efficient processing, enables developers to handle complex tasks with high speed and efficiency, boasting capabilities of processing over 75 tokens per second. With integration opportunities in Google Cloud's Vertex AI, as well as tools like Firebase and Colab, developers have the flexibility to incorporate the PaLM API into their existing frameworks. The API has already been utilized by companies like GameOn Technology, Vercel, and Wendy's to enhance user experiences and streamline operations, and Google is encouraging further innovation by offering public access to its documentation and prototype testing. At Google I/O 2023, Project Tailwind, an AI-driven notebook that enhances learning by leveraging personal notes, was showcased as an example of PaLM API's potential, highlighting the ease and speed with which developers can build powerful AI prototypes.

MediaPipe Solutions, introduced at Google I/O 2023, is a collection of on-device machine learning tools designed to streamline development for mobile, web, desktop, and IoT platforms. The suite includes MediaPipe Studio, MediaPipe Tasks, and MediaPipe Model Maker, offering no-code to low-code solutions for tasks such as audio classification, segmentation, and text embedding. Initially previewed in December 2022 with five tasks, the platform now boasts an additional nine, including Face Landmarker and Image Segmenter, with plans to expand further. MediaPipe Studio facilitates testing of MediaPipe-compatible models on the web, MediaPipe Tasks simplifies deployment across platforms with consistent code logic, and MediaPipe Model Maker allows for custom model training with minimal data. Developers can create versatile applications with ease, leveraging features like gesture recognition and the ability to customize models for specific needs. The ongoing development of MediaPipe Solutions aims to enhance accessibility and functionality, encouraging developers to explore and share their innovations.

ARCore's Geospatial Creator, powered by Google's augmented reality platform, enables developers and creators to design and publish world-anchored immersive experiences on 1.4 billion Android devices. The tool uses Photorealistic 3D Tiles from Google Maps Platform to allow users to visualize and place digital content in real-world environments. With new capabilities like Rooftop anchors, it simplifies the process of anchoring virtual content, supporting both Android and iOS devices. The platform facilitates cross-platform experiences, which can be shared without full app downloads, and has been used in collaborations with partners in various sectors, including gaming, retail, and tourism. Notable projects include TAITO's SPACE INVADERS: World Defense game, Gap's interactive storefront in Times Square, Global Street Art's augmented murals, and Singapore Tourism Board's immersive guided tour. The tool's versatility allows for the creation of applications that enhance community engagement, transform retail experiences, and provide innovative solutions for urban planning.

Google I/O 2023 highlighted Google's ongoing investments in AI, mobile, web, and cloud technologies to enhance developer productivity and innovation. The event showcased how generative AI is being integrated into Google's products to simplify workflows and boost efficiency. Key announcements included tools for building AI-powered products responsibly, enhancing mobile productivity through Modern Android Development, creating a more powerful and accessible web, and introducing new generative AI capabilities in the cloud for developers of all skill levels. Google is offering over 200 sessions and learning materials on its website, alongside global events such as Google I/O Connect and I/O Extended meetups to engage developers worldwide and foster collaboration and learning.

Google's ARCore, the largest cross-device augmented reality platform, now supports 1.4 billion Android devices and select iOS devices, and continues to evolve with new features designed to enhance immersive experiences. The platform introduced new capabilities such as the Streetscape Geometry API, Rooftop Anchors, Geospatial Depth, and Scene Semantics API, all aimed at improving interaction with real-world environments and providing precise anchoring and depth measurements. These tools enable developers to create transformative AR experiences by allowing virtual interactions with building geometry, enhancing depth perception for realistic occlusion, and utilizing AI to label scene components for customized user experiences. Additionally, Google released "Mega Golf," an open-source demo on GitHub, showcasing how these new APIs can be used in gaming, inviting developers to explore the potential of ARCore in creating realistic and engaging augmented reality applications.

In response to the growing complexity of development environments and the integration of machine learning with various platforms, Google AI has consolidated its machine learning products, tools, and guidance into a single destination at ai.google/build/machinelearning. This platform aims to streamline the process for developers at all levels by providing access to pre-trained datasets, the latest in Generative AI, and updates from major events like Google I/O. It also introduces new toolkits that offer end-to-end guidance for building AI solutions, combining open-source products and walkthroughs to help implement best practices. Initial toolkits focus on tasks such as building a text classifier with Keras and deploying a large language model on Android using Keras and TensorFlow Lite, with more resources promised in the future.

JetBrains and Google are exploring the potential of WebAssembly GC to enhance the performance and usability of multiplatform apps on the web, focusing on Kotlin. Traditionally, developers have had to either compile their app code to JavaScript or rewrite apps for web platforms, which often resulted in performance compromises. WebAssembly offers a promising alternative by allowing cross-platform code to run with near-native performance in browsers. Kotlin, a popular language for Android apps, is now being tested for web compatibility using WebAssembly, which preliminary results suggest could double the execution speed compared to JavaScript. The collaboration between browser vendors, academics, and service providers is crucial for overcoming technical challenges such as multi-language garbage collection and JavaScript interoperability. This initiative aims to harmonize web and native app development, allowing developers to streamline their workflow while leveraging the web's accessibility and native app performance, with further technical insights to be shared on the V8 Blog.

Google is enhancing the Google Pay button to improve the checkout experience by introducing a new look and customization options based on Material 3 design principles. The updated button is available in two versions to suit dark and light themed applications and includes a subtle stroke for better visibility in low contrast environments. The new API for Android allows developers to easily integrate and configure the button directly within their XML layout or programmatically, offering adjustments to properties like theme and corner radius, and automatically includes necessary graphics and translations. Additionally, the button now provides dynamic contextual information, such as the last four digits of the customer's most recent payment card, which has been shown to significantly boost transaction conversions. These updates are part of Google's efforts to offer a more consistent and informative checkout experience and will soon include support for Jetpack Compose to facilitate UI development with Kotlin APIs.

I/O FLIP is a Google-powered, AI-designed card game that showcases the capabilities of Google's new generative AI technologies, combining AI-generated custom character images and descriptions through DreamBooth on Muse and the PaLM API. Built using Flutter and Dart, the game offers an engaging user experience with features from the Flutter Casual Games Toolkit, while Cloud Run ensures scalability to accommodate numerous players. Players select character classes and powers to generate a pack of cards, create a team, and engage in best-of-3 matches, with elemental powers influencing gameplay dynamics. The game leverages Firebase for hosting and real-time interaction, maintaining security and integrity with App Check and Firebase security rules, while Dart Frog facilitates backend and frontend code sharing to prevent cheating and enhance development efficiency. The game encourages social interaction by allowing card sharing on platforms like Twitter and Facebook, and it is open-sourced to inspire developers to explore generative AI possibilities in their own products.

Testing plays a crucial role in software engineering, particularly in payment systems where even small errors can have major consequences. To address this, Google Pay has introduced an enhanced testing suite with Payment Service Provider (PSP) test cards, expanding the capabilities beyond the initial set of test cards. These new test cards, available in TEST mode, allow developers to create comprehensive end-to-end test scenarios using cards from popular payment processors like Stripe, Adyen, Braintree, WorldPay, and Checkout.com. This upgrade supports both manual and automated testing, enabling developers to validate a wide range of payment outcomes, including successful transactions, fraud, declines, and insufficient funds. Test automation is facilitated through familiar tools like UIAutomator and Espresso on Android, and these can be integrated into CI/CD workflows to bolster checkout experiences. Currently in beta, the new test suite is set to include web support later this year, reflecting Google Pay's commitment to providing developers with robust tools to enhance payment flow reliability and performance.

Creating a custom machine learning model involves several key steps and considerations, as outlined in the experience of developing a dog detection model for a web app using MediaPipe. The process begins with data preparation, which includes sourcing and annotating data, often requiring more time than anticipated and necessitating access to appropriate datasets. Simplifying the model is crucial, such as focusing on a limited number of classes and avoiding edge cases to streamline the training process. Iterative training and adjustments are expected, with the use of MediaPipe Model Maker facilitating quick updates through transfer learning. Prototyping the model outside of the main app, in environments like MediaPipe Studio, allows for rapid testing and iteration without the overhead of full app development cycles. Incremental changes, rather than sweeping modifications, can help fine-tune the model's performance, avoiding the introduction of new issues. By following these guidelines, developers can effectively customize models for tasks like image classification and object detection, enhancing their machine learning projects.

Developer Journey is a monthly series spotlighting diverse developers worldwide who share their experiences, challenges, and achievements. This edition features AiJing Wu, Jolina Li, and Maria Paz Muñoz Parra, who discuss their anticipation for Google I/O 2023, emphasizing its role as a platform for innovation, learning, and community engagement. Each developer highlights their favorite Google tools, such as Chrome, Google Colab, and accessibility tools, and shares insights into their projects, including web apps and recipe organizers. They also offer advice to newcomers in the developer community, stressing the importance of communication, goal-setting, and leveraging prior knowledge to succeed in technology. The series underscores the excitement surrounding Google I/O and encourages developers to connect and share their experiences using various Google tools and platforms.

Google I/O 2023, set to start on May 10, promises a range of sessions spotlighting advancements in Android development tools and multi-device experiences. The event will feature keynotes from influential figures like Tor Norbye, reflecting on the evolution of Android Studio and essential tools since its inception in 2013, and Sara Hamilton, who will discuss investments in multi-device experiences across the Android ecosystem. Dan Sandler will highlight Android's platform growth and emphasize creating quality apps with over 3 billion monthly active users as of 2021. Attendees can anticipate sessions on new features in Android development tools, Jetpack Compose, Kotlin updates, and the challenges of building apps for diverse screen sizes and types.