Bringing artworks to life with AR

Google Arts & Culture, in collaboration with the MediaPipe team, has developed Art Filter, a feature that utilizes augmented reality to bring culturally significant artworks to life on mobile devices. This innovative tool allows users to virtually try on famous art pieces, such as Frida Kahlo's Self-Portrait with Monkey or Vermeer's Girl with a Pearl Earring, by projecting high-resolution gigapixel images onto 3D meshes. The Art Filter employs MediaPipe's face mesh technology to create custom effects that align with the original style of the artworks, ensuring that objects are viewable from all angles. The feature processes user images entirely on-device, preserving privacy by not collecting biometric data, and uses Physically Based Rendering shaders to enhance the realism of materials like the Tengu helmet. Art Filter is part of Google Arts & Culture's broader initiative to use machine learning to enhance user engagement with global art, offering additional features such as Art Selfie and Art Transfer.