Analyzing and Classifying Data Format Strategies for Efficient Communication in Federated Learning

Federated learning (FL) is one of the most important technologies in the field of big data and artificial intelligence. It has the ability to balance the privacy of data preservation and enable collaborative training of a shared model. Although effective, Federated Learning presents critical challenges of high communication and computation overheads. This paper reviews recent approaches systemically to classify these two aspects and categorize data format strategies into four types based on the descriptiveness of data, level details and abstraction; gradients, parameter weights, knowledge distillation (KD), and federated feature learning (FFL). In addition, this paper evaluates these approaches regarding three criteria; communication reduction, computational cost, and accuracy. As the survey confirms, gradient and parameter weight methods are quite simple and have only a small computational price increase but, in return, offer relatively small communication reduction. In comparison, knowledge distillation (KD) and federated feature learning (FFL) achieve the highest efficient communication at the expense of increased computational complexity. Remarkably, from the point of view of losses in accuracy, the data compression and reduction in communication can be balanced. An analysis, that underlines the commitment for a standardized benchmark to be utilized in comparing these strategies and summarizes that all three criteria have to be taken into consideration. This depends on choosing an adequate strategy for the data format in federated learning (FL) applications.