18247447. COLLABORATIVE TRAINING WITH BUFFERED ACTIVATIONS (Rakuten Mobile, Inc.)
COLLABORATIVE TRAINING WITH BUFFERED ACTIVATIONS
Organization Name
Inventor(s)
Blesson Varghese of St Andrews (GB)
Rehmat Ullah of St Andrews (GB)
Peter Kilpatrick of Belfast (GB)
COLLABORATIVE TRAINING WITH BUFFERED ACTIVATIONS
This abstract first appeared for US patent application 18247447 titled 'COLLABORATIVE TRAINING WITH BUFFERED ACTIVATIONS
Original Abstract Submitted
Collaborative training with buffered activations is performed by partitioning a plurality of layers of a neural network model into a device partition and a server partition; transmitting, to a computation device, the device partition, training, collaboratively with the computation device through a network, the neural network model by applying the server partition to a set of activations to obtain a set of output instances, the set of activations obtained by one of receiving, from the computation device, the set of activations as output from the device partition, or reading, from an activation buffer, the set of activations as previously recorded, applying a loss function relating activations to output instances to each output instance among the current set of output instances to obtain a set of loss values, and computing a set of gradient vectors for each layer of the server partition based on the set of loss values.