18913134. ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS (Google LLC)
ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS
Organization Name
Inventor(s)
Noam M. Shazeer of Palo Alto CA US
Lukasz Mieczyslaw Kaiser of San Francisco CA US
Jakob D. Uszkoreit of Berlin DE
Niki J. Parmar of San Francisco CA US
Ashish Teku Vaswani of San Francisco CA US
ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS
This abstract first appeared for US patent application 18913134 titled 'ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS
Original Abstract Submitted
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output image. In one aspect, one of the methods includes generating the output image intensity value by intensity value according to a generation order of pixel-color channel pairs from the output image, comprising, for each particular generation order position in the generation order: generating a current output image representation of a current output image, processing the current output image representation using a decoder neural network to generate a probability distribution over possible intensity values for the pixel—color channel pair at the particular generation order position, wherein the decoder neural network includes one or more local masked self-attention sub-layers; and selecting an intensity value for the pixel—color channel pair at the particular generation order position using the probability distribution.