18211488. BACKPROPAGATION FOR DISCRETE VARIABLES (Microsoft Technology Licensing, LLC)

BACKPROPAGATION FOR DISCRETE VARIABLES

Organization Name

Microsoft Technology Licensing, LLC

Inventor(s)

Liyuan Liu of Redmond WA (US)

Chengyu Dong of San Diego CA (US)

Xiaodong Liu of Woodinville WA (US)

Bin Yu of Berkeley CA (US)

Jianfeng Gao of Woodinville WA (US)

BACKPROPAGATION FOR DISCRETE VARIABLES

This abstract first appeared for US patent application 18211488 titled 'BACKPROPAGATION FOR DISCRETE VARIABLES

Original Abstract Submitted

Generally discussed herein are devices, systems, and methods for backpropagation of a discrete latent variable. A method can include determining, to a second-order accuracy, an approximation of a gradient of a parameter of a discrete latent variable of a neural network (NN), adjusting the parameter based on the approximation of the gradient resulting in an adjusted parameter, and operating the NN using the adjusted parameter