Our Research on Membership Inference Attacks and Preventing Privacy Leaks - The JetBrains Blog

Membership inference attacks pose a significant threat to user privacy by exploiting machine learning models, particularly large language models (LLMs), to determine if specific data points were used during training without directly accessing the data. These attacks can reveal sensitive information, particularly in fine-tuned models, which are more vulnerable due to their smaller datasets and stronger memorization. The post by JetBrains researchers highlights the underestimated risk of these attacks and critiques current detection methods as inadequate. They introduce a new attack method called Error Zone Membership Inference Attack (EZ MIA), which focuses on the model's mistakes to detect membership signals more effectively than traditional methods. This approach requires fewer resources and offers a practical way to assess privacy risks in fine-tuned language models, particularly in differentiating between full and parameter-efficient fine-tuning like Low-Rank Adaptation (LoRA). The findings underscore the need for precise privacy auditing tools to prevent unintentional data exposure in LLMs, advocating for more focused research on privacy risks associated with fine-tuning practices.