NSF project CNS-2203412: MLWiNS: Distributed Learning over Multi-Access Channels: From Bandlimited Coordinate Descent to Gradient Sketching

Overview:

In many edge networks, edge devices are often connected to each other or a central node wirelessly. The unreliable nature of wireless connectivity, together with constraints in computing resources at edge devices, puts forth a significant challenge for the computation, communication and coordination required to learn an accurate model at the network edge. This project considers a many-to-one architecture for distributed learning over wireless multi-access channels , where multiple edge devices collaboratively train a high-dimensional machine learning model, using local data, in a distributed manner. In such a setting, the constraints in computing and communication resources dictate that the local updates at edge devices (senders) should be carefully crafted and compressed to make full use of the wireless bandwidth/power available and should work in concert with the edge server (receiver) so as to learn an accurate model.

Technical approaches and key outcomes:

Aiming to develop an integrated edge learning and wireless networking framework, this project will take a principled approach to investigate thoroughly two suites of edge learning algorithms tailored for wireless MAC channels, namely bandlimited coordinate descent and bandlimited gradient sketching. In these algorithms, sparsified or sketched versions of local updates are communicated using multi-carriers based transmissions where the number of subcarriers is constrained by the bandwidth; and each sender carries out power allocation across subcarriers based on gradient values and channel conditions in an integrated manner. Further, these edge learning algorithms will be devised using two intimately related methods:

1. First-order stochastic gradient descent methods,
2. Zero-order stochastic optimization methods.

Note that zero-order methods are derivative-free so they are very promising for wireless edge learning. In particular, because channel state information is essential to optimal power control and gradient estimation in the learning driven framework, we make dedicated effort to study machine learning aided channel estimation. We then pursue a comprehensive understanding of the impact of the wireless bandwidth/power on the accuracy and convergence of these algorithms. We evaluate the performance of the proposed learning framework for wireless edge networks, using simulation, experiments, and prototype development.

• This work promotes reliable and secure connected IoT systems, across data acquisition, across computation, and across communication.
• Success will tackle the fundamental challenge in the rapid deployment of AI methods, namely how to effectively carry out computation and coordination for edge learning while taking into account the constraints in wireless networks and computational resources at billions of edge devices.