AI Pill Identifier: Use Advanced RAG Strategies & FastSAM with YOLOv8

This project utilizes advanced artificial intelligence techniques, specifically NLP and computer vision, to assist healthcare by allowing users to take a photo of a pill and find information about it. The process is divided into different phases, including image segmentation with FastSAM and YOLOv8-seg, visual similarity with ResNet-18, text extraction and identification using GPT-4 vision, and advanced retrieval strategies with LlamaIndex. The system uses a user-friendly Gradio interface to display the results, which are divided into two columns: the three most similar images and three similar images that require attention due to their different descriptions from the input image. The project also explores various retrieval methods such as BM25 Retriever + Re-Ranking technique, Hybrid Fusion Retriever, and Advanced - Hybrid Retriever + Re-Ranking technique in LlamaIndex.