Telefonaktiebolaget lm ericsson (publ) (20240119305). ENERGY-EFFICIENT DEEP NEURAL NETWORK TRAINING ON DISTRIBUTED SPLIT ATTRIBUTES simplified abstract

ENERGY-EFFICIENT DEEP NEURAL NETWORK TRAINING ON DISTRIBUTED SPLIT ATTRIBUTES

Organization Name

telefonaktiebolaget lm ericsson (publ)

Inventor(s)

Selim Ickin of Stocksund (SE)

Konstantinos Vandikas of Solna (SE)

ENERGY-EFFICIENT DEEP NEURAL NETWORK TRAINING ON DISTRIBUTED SPLIT ATTRIBUTES - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240119305 titled 'ENERGY-EFFICIENT DEEP NEURAL NETWORK TRAINING ON DISTRIBUTED SPLIT ATTRIBUTES

Simplified Explanation

The abstract describes a method for operating a master node in a vertical federated learning system, where the neural network is split between workers and the master node. The method involves receiving layer outputs, determining missing outputs, generating imputed values, calculating gradients, splitting gradients, and transmitting them to workers.

• Receiving layer outputs from workers for a cut-layer in a split neural network.
• Determining if layer outputs for the cut-layer were not received from a worker.
• Generating imputed values for missing layer outputs.
• Calculating gradients for neurons in the cut-layer based on received and imputed outputs.
• Splitting gradients into groups for each worker.
• Transmitting groups of gradients to respective workers.

Potential Applications

This technology could be applied in various fields such as healthcare, finance, and telecommunications for collaborative machine learning tasks where data privacy is a concern.

Problems Solved

1. Addressing missing data from workers in a federated learning system. 2. Improving the efficiency and accuracy of training a split neural network in a distributed environment.

Benefits

1. Enhances data privacy by keeping sensitive information decentralized. 2. Optimizes training processes by handling missing data effectively. 3. Facilitates collaboration among multiple workers in a federated learning setup.

Potential Commercial Applications

Optimizing customer data analysis in retail industries using federated learning techniques.

Possible Prior Art

One possible prior art could be the use of distributed computing techniques in machine learning to handle missing data in a collaborative environment.

How does this method handle communication latency between the master node and workers?

The abstract does not provide details on how communication latency is managed in this method.

What security measures are in place to protect the imputed values during transmission to workers?

The abstract does not mention specific security measures implemented to protect imputed values during transmission.

Original Abstract Submitted

a method of operating a master node in a vertical federated learning, vfl, system including a plurality of workers for training a split neural network includes receiving layer outputs for a sample period from one or more of the workers for a cut-layer at which the neural network is split between the workers and the master node, and determining whether layer outputs for the cut-layer were not received from one of the workers. in response to determining that layer outputs for the cut-layer were not received from one of the workers, the method includes generating imputed values of the layer outputs that were not received, calculating gradients for neurons in the cut-layer based on the received layer outputs and the imputed layer outputs, splitting the gradients into groups associated with respective ones of the workers, and transmitting the groups of gradients to respective ones of the workers.