Bringing Street Murals to Life with Computer Vision

In Bedford-Stuyvesant, Brooklyn, a large mural titled "When Women Pursue Justice" celebrates 90 influential women and has become an integral part of the local art scene since its creation in 2005. Yuri Fukuda, an R&D Lead Senior Software Engineer living in Brooklyn, is leveraging computer vision technology to develop an augmented reality mobile application that allows users to identify and learn about the women featured in the mural. By capturing and labeling images of the mural from various perspectives, Yuri has successfully trained a custom object detection model using the YOLOv4 tiny Darknet model and TensorFlow Lite for on-device inference. The application currently identifies figures such as Dorothy Day, Audre Lorde, and Shirley Chisholm, and Yuri plans to expand the dataset to improve accuracy in different lighting conditions. Encouraging community participation, Yuri aims to inspire others to use technology to enrich public art and foster local engagement.