Linear Regression Over Encrypted Data With Homomorphic Encryption

In a tutorial developed as part of the Zama Bounty Program, the process of creating, training, and evaluating a LinearSVR regression model using the open-source Concrete ML library is detailed, with an emphasis on privacy-preserving techniques through Fully Homomorphic Encryption (FHE). The tutorial uses a single explanatory variable for simplicity, facilitating visualization of its relationship with the target variable, and employs a grid search to identify optimal hyperparameters for the LinearSVR model, including the inverse strength of the l2 penalization and the margin for support vectors. The process also involves quantization, which converts floating-point data to discrete integers, and compiling the quantized model for FHE to perform homomorphic inference. The tutorial compares the performance of models trained using Scikit-learn and Concrete ML, demonstrating that both methods yield similar results, with minimal performance differences attributed to the quantization and compilation processes. Moreover, the tutorial highlights the importance of providing a comprehensive data set during the compilation phase to ensure the FHE model's decision rule accurately generalizes beyond the observed data range.