January 2022 Summaries

Gretel's synthetic data is artificial data created from a machine learning model trained on a source dataset, replicating insights and correlations without memorizing original records. Gretel-synthetics uses LSTM neural networks to learn patterns in text or structured data. The outputs include a synthetic data model, initial synthetic data records, and premium reporting assessing quality and correlations with the original dataset. Differential privacy is used to protect sensitive data during training.

This study discusses the creation of high-quality synthetic time-series datasets for one of the largest financial institutions in the world using Gretel's methods. The temporal nature of time series data is useful for tracking and forecasting trends, but sharing such data between individuals and organizations can be challenging due to privacy concerns. By generating synthetic time-series data that are generalizable and shareable amongst diverse teams, financial institutions can gain a competitive edge and explore new opportunities. The bank's data science team provided a time series dataset containing customer account balance information over time. A pipeline was created to de-identify the time series dataset and then create a synthetic model that would generate an artificial dataset of the same size and shape. The accuracy of the data was assessed using a comparison of time series distributions for a district in the dataset, and the quality of the synthetic time series dataset was evaluated by fitting an ARIMA model to both the synthetic and original datasets. The privacy of the artificial data was then assessed by comparing the transformed and synthesized dataset to the original training dataset. Gretel's Similarity Privacy Filter removed all synthetic records that were duplicates of training records, providing strong privacy guarantees required to allow sharing inside a financial institution. The study demonstrated that Gretel's synthetic data can be as accurate, and in some cases even surpass that of real-world data used for machine learning classification tasks while maintaining high standards of privacy.

At the 35th Annual Conference on Neural Information Processing Systems, researchers from the Gretel research team highlighted several trends and takeaways related to machine learning and data privacy. These included advances in diffusion models for image and audio synthesis, improvements in language model performance for longer sequences, fine-tuning large pre-trained models, and addressing challenges with differential privacy (DP) such as balancing fairness and privacy, reducing worst-case privacy loss, and improving the efficiency of DP algorithms. Additionally, new datasets and metrics were introduced to improve data analysis and evaluation in supervised learning tasks.

The use of Gretel's Privacy Filters on synthetic datasets can impact machine learning accuracy in various ways. These filters prevent the creation of synthetic data with weaknesses commonly exploited by adversarials, thus enhancing privacy protection. Experiments conducted using these filters showed that synthetic data average accuracy is usually not far from original data's average accuracy. When the Privacy Filters are set to "med", the accuracy remains similar and sometimes even exceeds the original data accuracy. When set to "high", the results are variable but still quite good, with a modest hit on accuracy in some cases, no impact in others, and sometimes an improvement in accuracy. The tradeoff between privacy and accuracy is complex, as making data private can lead to more people sharing their data and being more honest about it, which also impacts accuracy.