Represent Chess Boards Digitally with Computer Vision

Shai Nisan, Ph.D., shares his journey of developing an application that uses computer vision to convert images of real-life chess games into a digital format, allowing players to save, share, and analyze their games. Inspired by his son's request, Nisan tackled the problem by breaking it into two sub-problems: recognizing the chessboard and identifying the chess pieces. He used YOLOv8 for detecting chessboard corners and trained a model on a dataset augmented with Roboflow to improve accuracy. By employing techniques such as corner and piece recognition, perspective transformation, and intersection over union calculations, he successfully transformed board images into Forsyth–Edwards Notation (FEN) data. The method, which achieved over 99% accuracy, can be generalized to different chessboards, offering a practical, lightweight solution that avoids resource-intensive processes. This application enhances the user experience by enabling them to interact with chess engines and enjoy digital chess tools, fulfilling the initial request from his son.