Google llc (20240193926). ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS simplified abstract

ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS

Organization Name

google llc

Inventor(s)

Noam M. Shazeer of Palo Alto CA (US)

Lukasz Mieczyslaw Kaiser of San Francisco CA (US)

Jakob D. Uszkoreit of Berlin (DE)

Niki J. Parmar of San Francisco CA (US)

Ashish Teku Vaswani of San Francisco CA (US)

ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240193926 titled 'ATTENTION-BASED IMAGE GENERATION NEURAL NETWORKS

The patent application describes methods, systems, and apparatus for generating an output image using a decoder neural network with local masked self-attention sub-layers.

• The method involves generating the output image intensity value based on a generation order of pixel-color channel pairs.
• For each position in the generation order, a current output image representation is generated and processed using the decoder neural network to determine a probability distribution of possible intensity values.
• An intensity value for the pixel-color channel pair at that position is then selected based on the probability distribution.

Potential Applications:

• Image processing and generation in various industries such as entertainment, design, and healthcare.
• Enhancing image quality and resolution in digital photography and video editing applications.

Problems Solved:

• Efficient generation of high-quality output images with accurate intensity values.
• Streamlining the image processing workflow by automating the selection of intensity values.

Benefits:

• Improved image generation accuracy and quality.
• Time and cost savings in image processing tasks.
• Enhanced user experience in applications requiring image generation.

Commercial Applications:

• Digital art creation software.
• Medical imaging systems.
• Video game development tools.

Questions about the technology: 1. How does the decoder neural network with local masked self-attention sub-layers improve image generation accuracy? 2. What are the potential limitations of using this method in real-time image processing applications?

Original Abstract Submitted

methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating an output image. in one aspect, one of the methods includes generating the output image intensity value by intensity value according to a generation order of pixel-color channel pairs from the output image, comprising, for each particular generation order position in the generation order: generating a current output image representation of a current output image, processing the current output image representation using a decoder neural network to generate a probability distribution over possible intensity values for the pixel-color channel pair at the particular generation order position, wherein the decoder neural network includes one or more local masked self-attention sub-layers; and selecting an intensity value for the pixel-color channel pair at the particular generation order position using the probability distribution.