Whispered Tuning: Data Privacy Preservation in Fine-Tuning LLMs through Differential Privacy
- 1 School of Computer Science Engineering & Technology, Bennett University, Greater Noida, India
- 2 School of Computer Science Engineering & Technology, Bennett University, Greater Noida, India
- 3 School of Cybersecurity and Privacy, Georgia Institute of Technology, Atlanta, USA
- 4 Cloudemy Technology Labs, Chandigarh, India
Abstract
The proliferation of Large Language Models (LLMs) across various sectors underscored the urgency of addressing potential privacy breaches. Vulnerabilities, such as prompt injection attacks and other adversarial tactics, could make these models inadvertently disclose their training data. Such disclosures could compromise personal identifiable information, posing significant privacy risks. In this paper, we proposed a novel multi-faceted approach called Whispered Tuning to address privacy leaks in large language models (LLMs). We integrated a PII redaction model, differential privacy techniques, and an output filter into the LLM fine-tuning process to enhance confidentiality. Additionally, we introduced novel ideas like the Epsilon Dial for adjustable privacy budgeting for differentiated Training Phases per data handler role. Through empirical validation, including attacks on non-private models, we demonstrated the robustness of our proposed solution SecureNLP in safeguarding privacy without compromising utility. This pioneering methodology significantly fortified LLMs against privacy infringements, enabling responsible adoption across sectors.
- Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288.
- Liu, Y., Deng, G.L., Li, Y.K., Wang, K.L., Zhang, T.W., Liu, Y.P., Wang, H.Y., Zheng, Y. and Liu, Y. (2023) Prompt Injection Attack against LLM-Integrated Applications. arXiv preprint arXiv:2306.05499.
- Shi, W., Shea, R., Chen, S., Zhang, C., Jia, R. and Yu, Z. (2022) Just Fine-Tune Twice: Selective Differential Privacy for Large Language Models. In: Goldberg, Y., Kozareva, Z. and Zhang, Y., Eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Abu Dhabi, 6327-6340. https://doi.org/10.18653/v1/2022.emnlp-main.425
- Klymenko, O., Meisenbacher, S. and Matthes, F. (2022) Differential Privacy in Natural Language Processing: The Story So Far. In: Feyisetan, O., Ghanavati, S., Thaine, P., Habernal, I. and Mireshghallah, F., Eds., Proceedings of the Fourth Workshop on Privacy in Natural Language Processing, Association for Computational Linguistics, Seattle, 1-11. https://doi.org/10.18653/v1/2022.privatenlp-1.1
- Behnia, R., Ebrahimi, M.R., Pacheco, J. and Padmanabhan, B. (2022) EW-Tune: A Framework for Privately Fine-Tuning Large Language Models with Differential Privacy. 2022 IEEE International Conference on Data Mining Workshops (ICDMW), Orlando, FL, 28 November-1 December 2022, 560-566. https://doi.org/10.1109/ICDMW58026.2022.00078
- Li, Y., Tan, Z. and Liu, Y. (2023) Privacy-Preserving Prompt Tuning for Large Language Model Services. arXiv preprint arXiv:2305.06212.
- Plant, R., Giuffrida, V. and Gkatzia, D. (2022) You Are What You Write: Preserving Privacy in the Era of Large Language Models. arXiv preprint arXiv:2204.09391. https://doi.org/10.2139/ssrn.4417900
- Kandpal, N., Wallace, E. and Raffel, C. (2022) Deduplicating Training Data Mitigates Privacy Risks in Language Models. International Conference on Machine Learning, Baltimore, Maryland, July 17-23 2022, 10697-10707.
- Yu, D., Kamath, G., Kulkarni, J., Yin, J., Liu, T.-Y. and Zhang, H.S. (2022) Per-Instance Privacy Accounting for Differentially Private Stochastic Gradient Descent. arXiv preprint arXiv:2206.02617.
- Mireshghallah, F., Uniyal, A., Wang, T.H., Evans, D. and Berg-Kirkpatrick, T. (2022) Memorization in NLP Fine-Tuning Methods. arXiv Preprint arXiv:2205.12506.