This new method for measuring dataset difficulty uses class-wise autoencoders, which are specialized deep networks trained to reconstruct their own class of data. The reconstruction error alone is not sufficient, so a metric called Reconstruction Error Ratios (RERs) is introduced as a way to serve this purpose. RERs provide a quantitative measure of classification difficulty without needing to train a full-scale classifier. They can be computed quickly using shallow autoencoders trained on feature representations from a foundation model like CLIP or DINOv2. The method strongly correlates with classifier error rates, making it a powerful tool for dataset analysis. It also offers insights into dataset quality, including estimating the potential improvement from collecting more data versus the inherent limits imposed by class structure. This research opens exciting possibilities for data-centric AI, such as benchmarking datasets before model training, efficiently curating high-quality datasets, and identifying and correcting mislabeled samples.