August 2020 Summaries

Instant Motion Tracking, a new solution released in MediaPipe, enhances Augmented Reality (AR) experiences by allowing seamless placement of virtual 2D and 3D content on both static and moving surfaces without requiring initialization or calibration, utilizing six degrees of freedom tracking. This technology builds on the previously released MediaPipe Box Tracking solution and operates by combining 3D camera translation and rotation data, derived from visual signals and an IMU sensor, to create immersive AR interactions. An open-source Android application demonstrates its capabilities, enabling users to place virtual objects and animations in real-world environments simply by tapping the camera viewfinder. The solution's ease of integration into various systems is facilitated by MediaPipe's cross-platform framework, making it accessible for devices with camera and IMU functionality. The technology is already being leveraged by PixelShift.AI, a startup focused on video content creation, to craft mixed reality experiences for VTubers, with plans to expand its availability to the broader community.

On August 28, 2020, Google announced a security measure to block sign-ins from embedded browser frameworks to protect against "man-in-the-middle" phishing attacks, effective January 4, 2021. This move affects applications using the Chromium Embedded Framework (CEF) and other unsupported browsers, necessitating developers to implement OAuth 2.0 flows via supported browsers or full native browsers to ensure continued functionality. Google emphasizes that modern browsers with security updates and certain standards, such as having JavaScript enabled and not using automation features, will continue to be supported. Developers are encouraged to test for compatibility by adding specific HTTP headers to their requests and are directed to previous resources for more information on preventing phishing attacks.

The ML Kit Pose Detection API is a new feature added to Google's ML Kit, designed to facilitate the integration of on-device machine learning into mobile apps, enhancing user experiences by tracking physical actions in real-time. Compatible with both Android and iOS, this lightweight solution offers a 33-point skeletal match that includes facial landmarks and tracks hands and feet, making it ideal for applications such as fitness and yoga trackers. Utilizing the BlazePose pipeline, the API offers two operating modes—"Fast" for high frame rates and "Accurate" for stable coordinates—and supports both video and static image use cases. It optimizes performance through a two-step detection process, combining face detection with a prominent person detection algorithm, and includes a streaming mode for further latency improvements. Early adopters like Groovetime and adidas Training are using the API to provide real-time feedback on dance moves and virtual workouts, respectively, highlighting its potential in various real-world applications.

Google Assistant Developer Day, held virtually on October 8, 2020, offered developers a platform to explore new tools and features designed to enhance user engagement with Google Assistant on mobile apps and Smart Displays. The event, hosted by Google product management directors Baris Gultekin and Payam Shodjai, provided insights into best practices for integrating Google Assistant to facilitate voice interactions with apps. Attendees gained the opportunity to interact with Google leaders and engineers and attend sessions focused on native development and discoverability via voice. Notable guest speakers, including industry leaders from Postmates, CoolGames, and MyFitnessPal, shared their experiences of how voice technology has reshaped user interaction with their services. This free event encouraged developers to leverage the new tools for creating engaging content that caters to users who prefer voice commands, with registration available through the event website.

Eustache Luckens Yadley, a web developer from Port-au-Prince, Haiti, has dedicated his career to creating web applications that support the local economy, amidst rising unemployment rates in the country. Observing a lack of reliable delivery methods for home purchases, Yadley developed "Livrezonpam", an app that connects companies needing delivery services with available workers. By leveraging the resources and expertise available through the Port-au-Prince Google Developer Group, Yadley refined his app using Google Cloud Platform and Google Maps Platform API to enhance user experience. His efforts have contributed to job creation and improved local business operations, with "Livrezonpam" continuing to grow on the Google Play App Store, while Yadley remains active in the developer community to further advance his skills and impact.

Spotlight, a new global program launched by Women Techmakers on July 9, 2020, aims to amplify the stories and career paths of women in technology, offering guidance and advice to help women navigate their tech careers. The series was inspired by community demand for career navigation support, and it leverages an extensive network of professional women from diverse backgrounds in the tech industry. Featuring weekly episodes, the program covers themes such as job selection, success in tech roles, overcoming imposter syndrome, and leading with confidence, with each episode showcasing women with varied experiences, from engineering graduates to coding bootcamp attendees. The program's first episode featured Priyanka Vergadia, a Developer Advocate for Google Cloud, who discussed risk-taking and dealing with failure, and since its launch, the response has been overwhelmingly positive. Spotlight continues to expand its library of career advice, offering inspiration and practical insights to viewers worldwide.

G Suite Add-ons enhance user productivity by integrating external application functionalities directly into the G Suite environment with a persistent, context-aware sidebar. Traditionally built using Apps Script, G Suite is now offering developers the flexibility to use alternate runtimes, such as Node.js, without adding new capabilities but allowing for more personalized tech stack choices. While Apps Script provides a straightforward, serverless approach with easy access to Google APIs, alternate runtimes require more setup but offer increased testability and integration with existing development tools and processes. Developers like Shailesh Matariya from Gfacility have reported significantly improved performance and maintainability using alternate runtimes, with faster data loading and enhanced workflow management. This flexibility supports both prototyping simpler add-ons with Apps Script and developing more complex systems with alternate runtimes, aligning with developers' existing skills and infrastructure.

ChromeOS.dev is introduced as a comprehensive resource for developers aiming to create world-class apps and games for Chrome OS, providing technical documentation, code samples, and guidelines for web, Android, and Linux developers. With the rise in Chromebook sales during early 2020, driven by increased home use, the platform offers tools and best practices to enhance app experiences on larger screens. It features updates like a customizable Linux Terminal, Android Emulator support on Chromebooks, and improved Project Wizard in Android Studio for better app development. ChromeOS.dev, available in multiple languages, is an open-source platform offering offline access to resources to streamline the development process and optimize apps for various Android devices, including tablets and foldables. The initiative underscores Chrome OS's commitment to supporting developers at all levels to create secure, efficient, and engaging applications across platforms.

Cast Connect is a new feature introduced by Google that enhances the integration of Google Cast with Android TV apps, allowing users to cast content directly to their Android TV app while maintaining control from their sender app. This development combines the benefits of Android TV's immersive playback experience with the flexibility of controlling content via Android, iOS, or Chrome apps, and now additionally with the Android TV remote. This improvement facilitates easier engagement with content directly on the TV without solely relying on mobile devices. Google has collaborated with partners like CBS and Stan to launch Cast Connect, with the library now available on Android, iOS, and Chrome. Developers interested in incorporating Cast Connect into their frameworks can access resources on the Google Cast Developers site, with support available from the Cast SDK team and the developer community on Stack Overflow.

Google's ML Kit has expanded its capabilities with the introduction of the Digital Ink Recognition API for Android and iOS, enabling developers to create applications where stylus and touch inputs are prioritized. Unlike previous ML Kit APIs that focus on text or images, this new API interprets user strokes on screens to recognize handwriting and drawings, the same technology used in Google’s Gboard, Quick, Draw!, and AutoDraw experiments. The API supports over 300 languages and 25 writing systems, including major languages and diverse scripts like Chinese, Japanese, and Arabic. It features classifiers for text, shapes, and emojis, and operates entirely on-device without requiring an internet connection, though models must be previously downloaded and are approximately 20MB in size. The recognition process is efficient, with typical text recognition taking about 100 ms on standard mobile devices. Developers interested in implementing this technology can refer to the documentation and sample apps for guidance.

James Ward, a Developer Advocate, shares his experience of migrating his blog from WordPress to Hugo, a static site generator, and outlines the process of setting up automatic deployment for both live and draft versions of the site using Firebase Hosting and Cloud Run. By leveraging Hugo's ability to use plain files stored in GitHub, Ward describes the creation of a continuous delivery pipeline with Cloud Build, facilitating the deployment of production changes to Firebase Hosting and draft reviews on Cloud Run. The setup involves configuring Cloud Build triggers, managing permissions for Cloud Run and Firebase Hosting, and handling theme changes through git submodules, allowing for seamless deployment of updates and draft viewing across different branches. He provides detailed instructions for both setting up new and existing Hugo projects, highlights the importance of version control and testing, and encourages feedback from those who encounter challenges in implementing similar setups.