September 2021 Summaries

This article provides a tutorial on how to create a computer vision dataset for training a Computer Vision (CV) model using the Hierarchical Data Format version 5 (HDF5) file format and comparing it with Hub, an HDF5 alternative. The HDF5 format is popular for managing large datasets but may not be optimized for deep learning tasks. In contrast, Hub offers a deep learning-native dataset format that can integrate with machine learning frameworks like Pytorch or Tensorflow. By using the appropriate format, creating computer vision datasets can significantly impact the success of a deep learning project.

Computer vision, a challenging area in machine learning, involves teaching computers to distinguish visual representations, a task humans naturally excel at. This article explores image embeddings, which are dense vector representations of images used in various tasks like classification. Using a convolutional neural network (CNN), specifically a pre-trained ResNet-18 model, image embeddings are generated from a Kaggle Dog Breed Images dataset. These embeddings represent images in lower-dimensional space, making them crucial for tasks such as creating search engines based on image similarity. The process involves extracting features up to the penultimate layer of the CNN, with embeddings stored in Activeloop Deep Lake for easy reuse. The embeddings enable efficient classification and similarity searches by analyzing cosine similarity between vectors. The dataset, containing 918 images of different dog breeds, is processed using PyTorch, and the embeddings are uploaded to Deep Lake for further use. This approach highlights the utility of embeddings in computer vision applications across various industries, demonstrating how such techniques can enhance search and classification tasks.

The article compares the time taken to upload a computer vision dataset to Amazon Web Service (AWS) s3 bucket and Hub, with the aim of identifying the fastest method. It uses a large-scale fish segmentation and classification dataset from Kaggle for benchmarking. The results show that using AWS CLI is faster than boto3, but uploading the entire dataset to Hub using parallel computing was 2 times faster than AWS CLI and ~20 times faster than boto3. This indicates that Hub can significantly speed up the data preparation stage in a Machine Learning workflow.