August 2020 Summaries

The text describes a video search system that uses image similarity to retrieve videos from a repository. It explains the process of converting videos into embeddings, which involves extracting key frames and converting their features into vectors. The workflow includes importing videos using OpenCV library, cutting each video into frames, and inserting extracted vectors (embeddings) into Milvus. For searching, it uses the same VGG model to convert input images into feature vectors and inserts them into Milvus to find similar vectors. It then retrieves corresponding videos from Minio based on Redis correlations. The article also provides a sample dataset of 100,000 GIF files from Tumblr for building an end-to-end solution for video search. Deployment steps are outlined using Docker images and docker-compose.yml configuration file. Finally, the system's interface is displayed, allowing users to input target images and retrieve similar videos.

The second-generation search-by-image system uses CNN + Milvus solution. Feature extraction is done using convolutional neural network (CNN) as the underlying technology. VGG16 model is used for image feature extraction, and Keras + TensorFlow are utilized for technical implementation. Milvus, an open-source vector search engine, is employed to store and manage feature vectors, calculate similarity, and return vector data in the nearest neighbor range. The system also includes image processing techniques such as normalization, bytes conversion, and black border removal.

Yupoo Picture Manager, which manages tens of billions of images for its users, has an urgent need to quickly locate images within its growing gallery. To address this issue, the company developed a search by image service that underwent two evolutions. The first-generation system used Perceptual hash (pHash) algorithm for feature extraction and ElasticSearch for similarity calculation. However, it had limitations in handling images with altered integrity. The second-generation system introduced a new underlying technology to overcome these limitations.