刘佳伟,马本生,王志芳

• 1:黑龙江大学电子工程学院

摘要(Abstract)：

在联邦学习采用差分隐私保护机制的过程中，梯度裁剪与噪声添加不可避免地造成模型损失函数形态尖锐化、鲁棒性降低以及准确率显著下降。为此，提出了一种基于锐度感知最小化优化器(Sharpness-aware minimizer, SAM)的差分隐私联邦学习算法(Federated learning with differential privacy based on sharpness aware minimization optimizer, DP-Fedrsam),从客户端训练阶段协同优化隐私保护强度与模型精度。在客户端本地训练中，引入锐度感知最小化优化器以寻找平坦损失区域，提升隐私保护与模型效用间的平衡能力；同时，采用对称交叉熵损失函数，缓解客户端模型对局部最优解地过拟合，降低对全局收敛目标的偏离，增强在噪声干扰下的鲁棒性，加快训练收敛过程，并提升模型的整体稳定性。在Fashion-MNIST和CIFAR-10数据集上的实验表明，所提出方法相比现有算法，在测试准确率和隐私保护强度方面均具备更优的综合性能。

关键词(KeyWords)： 差分隐私;联邦学习;锐度感知最小化优化器;对称交叉熵损失函数

Abstract:

Keywords:

基金项目(Foundation): 黑龙江省自然科学基金资助项目(PL2024F026,PL2024F027)

作者(Author): 刘佳伟,马本生,王志芳

DOI: 10.13482/j.issn1001-7011.2026.04.004

参考文献(References)：

• [1] DWORK C,MCSHERRY F,NISSIM K,et al.Calibrating noise to sensitivity in private data analysis [C].Proceedings of the Theory of Cryptography:Third Theory of Cryptography Conference.New York:Springer,2006:265-284.
• [2] BASSO T,MATSUNAGA R,MORAES R,et al.Challenges on anonymity,privacy,and big data [C].Proceedings of the 2016 Seventh Latin-American Symposium on Dependable Computing.Colombia:IEEE Press,2016:164-171.
• [3] ZHA X.Research on anew attribute encryption anonymous algorithm in cloud computing environment [C].Proceedings of the Journal of Physics:Conference Series.Guangzhou:IOP Publishing,2020,1673(1):012059.
• [4] ZHANG Y,LIU N,WANG S.A differential privacy protecting K-means clustering algorithm based on contour coefficients [J].Plas One,2018,13(11):e0206832.
• [5] HUANG L,ZHOU J,LIN J,et al.View analysis of personal information leakage and privacy protection in big data era-based on Q method [J].Aslib Journal of Information Management,2022,74(5):901-927.
• [6] KUMARASINGHE U,NABEEL M,DE ZOYSA K,et al.HeteroGuard:defending heterogeneous graph neural networks against adversarial attacks [C].Proceedings of the 2022 IEEE International Conference on Data Mining Workshops.Orlando:IEEE Press,2022:698-705.
• [7] MCMAHAN B,MOORE E,RAMAGE D,et al.Communication-efficient learning of deep networks from decentralized data [C].Proceedings of the 20th International Conference on Artificial Intelligence and Statistics.Fort Lauderdale:PMLR,2017:1273-1282.
• [8] KARIMIREDDY S P,KALE S,MOHRI M,et al.Scaffold:stochastic controlled averaging for federated learning [C].Proceedings of the International Conference on Machine Learning.Palermo:PMLR,2020:5132-5143.
• [9] LI T,SAHU A K,ZAHEER M,et al.Federated optimization in heterogeneous networks [C].Proceedings of Machine Learning and Sstems,2020,2:429-450.
• [10] GEYER R C,KLEIN T,NABI M.Differentially private federated learning:a client level perspective [J].arXiv preprint arXiv:1712.07557,2017.
• [11] ZHENG Q,CHEN S,LONG Q,et al.Federated f-differential privacy [C].Proceedings of the International Conference on Artificial Intelligence and Statistics.Virtual Event:PMLR,2021:2251-2259.
• [12] TRIASTCYN A,FALTINGS B.Federated learning with bayesian differential privacy [C].Proceedings of the IEEE International Conference on Big Data.Los Angeles:IEEE Press,2019:2587-2596.
• [13] NOBLE M,BELLET A,DIEULEVEUT A.Differentially private federated learning on heterogeneous data [C].Proceedings of the International Conference on Artificial Intelligence and Statistics.Virtual Event:PMLR,2022:10110-10145.
• [14] YANG Q,LIU Y,CHEN T,et al.Federated machine learning:Concept and applications [J].ACM Transactions on Intelligent Systems and Technology,2019,10(2):1-19.
• [15] DWORK C.Differential privacy:a survey of results [C].Proceedings of the International conference on theory and applications of models of computation.Berlin:Springer Berlin Heidelberg,2008:1-19.
• [16] DWORK C,ROTH A.The algorithmic foundations of differential privacy [J].Foundations and Trends?? in Theoretical Computer Science,2014,9(3-4):211-407.
• [17] WANG Y,MA X,CHEN Z Y,et al.Symmetriccross entropy for robust learning with noisy labels [C].Proceedings of the IEEE/CVF International Conference on Computer Vision,Seou:IEEE Press,2019:284-293.
• [18] MAO A,MOHRI M,ZHONG Y.Cross-entropy loss functions:theoretical analysis and applications [C].Proceedings of the International Conference on Machine Learning.Hawaii:PMLR,2023:23803-23828.
• [19] BU Y,ZOU S,LIANG Y,et al.Estimation of KL divergence:optimal minimax rate [J].IEEE Transactions on Information Theory,2018,64(4):2648-2674.
• [20] ZHANG Y,HE H,ZHU J,et al.On the duality between sharpness-aware minimization and adversarial training [J].arXiv preprint arXiv:2402.15152,2024.
• [21] FU J,CHEN Z,HAN X.dap dp-fl:Differentially private federated learning with adaptive noise [C].Proceedings of the 21st IEEE International Conference on Trust,Security and Privacy in Computing and Communications.Wuhan:IEEE Press,2022:656-663.