Measuring What Matters: Objective Metrics for Image Generation Assessment

Generating high-quality visuals with advanced models has become increasingly accessible, transforming industries like advertising and gaming. While creating images is relatively straightforward, assessing their quality is complex due to subjective human biases and varying definitions of quality. Pruna introduces objective metrics to evaluate aspects like quality, coherence, and originality, using single and pairwise modes for absolute and relative evaluations. The metrics are categorized into efficiency, which measures speed and resource use, and quality, which assesses intrinsic image quality and alignment with prompts. Distribution alignment metrics like Fréchet Inception Distance (FID) and Clip Maximum-Mean-Discrepancy (CMMD) measure how closely generated images resemble real-world distributions. Prompt alignment metrics, such as CLIPScore, evaluate the semantic match between images and text prompts. Perceptual alignment metrics, including PSNR, SSIM, and LPIPS, focus on pixel-level and feature-level similarities. Each metric captures different aspects of image quality, making them suitable for various scenarios, and Pruna's open-source framework allows customization and contribution from the community.