September 2021 Summaries

Natural language processing (NLP) has advanced significantly over the past decade, providing immense opportunities for synthetic text generation through large language models, such as word2vec and BERT. However, these models come with ethical and environmental concerns, including the risk of perpetuating societal biases present in training data and the high carbon footprint associated with their development. Gretel, a company exploring NLP innovations, emphasizes the need for responsible usage of these models by considering privacy implications and curating unbiased datasets, although achieving the latter remains a challenging research problem. Despite these challenges, Gretel aims to democratize access to NLP technology by offering users efficient ways to generate high-quality synthetic text while balancing ethical considerations. The company is developing new metrics to evaluate text quality and plans to address these issues further in an upcoming blog series.

Gretel Synthetic has introduced new privacy protection mechanisms to enhance the safety of synthetic data. These mechanisms counter various adversarial attacks, including Membership Inference, Attribute Inference, Memorization Attacks, Model Inversion, and more. The company offers Overfitting Prevention, Similarity Filters, Outlier Filters, and Differential Privacy as privacy protection options. Users can configure these mechanisms through a configuration file to achieve desired levels of privacy protection based on their data sharing use case.