18149682. SKETCHED AND CLUSTERED FEDERATED LEARNING WITH AUTOMATIC TUNING simplified abstract (International Business Machines Corporation)
Contents
SKETCHED AND CLUSTERED FEDERATED LEARNING WITH AUTOMATIC TUNING
Organization Name
International Business Machines Corporation
Inventor(s)
Arpan Mukherjee of West Bengal (IN)
Georgios Kollias of White Plains NY (US)
Theodoros Salonidis of Wayne PA (US)
Shiqiang Wang of White Plains NY (US)
SKETCHED AND CLUSTERED FEDERATED LEARNING WITH AUTOMATIC TUNING - A simplified explanation of the abstract
This abstract first appeared for US patent application 18149682 titled 'SKETCHED AND CLUSTERED FEDERATED LEARNING WITH AUTOMATIC TUNING
Simplified Explanation
The abstract describes a method for automatic adaptive client selection in federated learning, where a server sends machine learning model parameters to clients, receives sketches of gradients computed by clients, computes similarity between clients, clusters clients based on similarity, optimizes client clusters and sketch dimensions, selects a subset of clients to send gradients, and aggregates the gradients.
- Server sends machine learning model parameters to clients
- Clients compute gradients using parameters and send sketches of gradients to server
- Server computes similarity between clients and clusters them
- Server optimizes client clusters and sketch dimensions based on memory consumption, communication overhead, and performance metric
- Server selects subset of clients from client clusters to send gradients
- Server aggregates gradients sent by subset of clients
---
- Potential Applications
- Federated learning systems
- Machine learning model training
- Distributed computing environments
- Problems Solved
- Efficient client selection in federated learning
- Optimization of communication overhead and memory consumption
- Improved performance in distributed machine learning
- Benefits
- Reduced communication overhead
- Optimal client selection for gradient computation
- Enhanced performance in federated learning systems
Original Abstract Submitted
A computer-implemented method, a computer program product, and a computer system for automatic adaptive client selection in federated learning. A server sends parameters of a machine learning model to all of clients, where all of the clients compute respective gradients using the parameters. The server receives sketches of the respective gradients, where the sketches are computed by all of the clients. The server uses the sketches to compute similarity between all of the clients and clusters the all of the clients based on the similarity. The server optimizes a number of client clusters and a dimension of the sketches, subject to a constraint of memory consumption, a constraint of communication overhead, and a performance metric. The server determines a subset of the clients that send the respective gradients, by selecting the clients from the client clusters. The server aggregates the respective gradients sent by the subset of the clients.
