论文标题

无线联合学习中有效的无线模型聚合的时间相关的稀疏

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

论文作者

Sun, Yuxuan, Zhou, Sheng, Niu, Zhisheng, Gündüz, Deniz

论文摘要

联合边缘学习(Feel)是一个有希望的分布式机器学习(ML)框架,可推动Edge Intelligence应用程序。但是,由于动态的无线环境和边缘设备的资源限制,通信成为主要的瓶颈。在这项工作中,我们提出了与混合聚合(TCS-H)相关的稀疏性,以进行沟通效率的感觉,从而利用了模型压缩和空中计算的共同利用。通过利用模型参数之间的时间相关性,我们构建了一个全局的稀疏掩码,该掩模在跨设备之间是相同的,因此可以启用有效的模型聚合直播。每个设备进一步构建了局部稀疏向量,以探索其自己的重要参数,这些参数是通过数字通信与正交多重访问来汇总的。我们进一步设计了TCS-H的设备调度和电源分配算法。实验结果表明,在有限的通信资源下,与常规的TOP-K稀疏相比,TCS-H可以达到明显更高的准确性,而正交模型聚集则可以达到I.I.D。和non-i.i.d。数据分布。

Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源