Making FHE Faster for ML: Beating our Previous Paper Benchmarks with Concrete ML

Zama has made significant advancements in speeding up Fully Homomorphic Encryption (FHE) for machine learning by developing Concrete ML, which has surpassed previous performance benchmarks. Initially, Zama used an ONNX-based compiler prototype to convert neural networks into FHE-friendly models, demonstrating the potential of the TFHE cryptographic scheme for handling deep neural networks. However, the complexity of integrating machine learning and cryptography in a single compiler led to a strategic pivot towards a more generic framework using MLIR, which allowed for better support of hardware accelerators and improved system efficiency. This transition enabled a separation of machine learning tasks from cryptographic ones, focusing on cryptographic security while letting users manage machine learning-specific choices. The new approach, utilizing the Concrete ML and Concrete Python frontends, offers flexible support for various backends and has achieved faster execution times for neural network models such as NN-20 and NN-50 compared to earlier methods. Zama's continued efforts in improving the TFHE-rs library, leveraging the MLIR framework, and enhancing quantization techniques have contributed to these improvements, making FHE more accessible and efficient for real-world applications.