Google llc (20250005798). PROCESSING IMAGES USING SELF-ATTENTION BASED NEURAL NETWORKS

PROCESSING IMAGES USING SELF-ATTENTION BASED NEURAL NETWORKS

Organization Name

google llc

Inventor(s)

Neil Matthew Tinmouth Houlsby of Zurich CH

Sylvain Gelly of Aix-en-Provence FR

Jakob D. Uszkoreit of Berlin DE

Xiaohua Zhai of Zurich CH

Georg Heigold of Aachen DE

Lucas Klaus Beyer of Zurich CH

Alexander Kolesnikov of Zurich CH

Matthias Johannes Lorenz Minderer of Zurich CH

Dirk Weissenborn of Berlin DE

Mostafa Dehghani of Amsterdam NL

Alexey Dosovitskiy of Berlin DE

Thomas Unterthiner of Berlin DE

PROCESSING IMAGES USING SELF-ATTENTION BASED NEURAL NETWORKS

This abstract first appeared for US patent application 20250005798 titled 'PROCESSING IMAGES USING SELF-ATTENTION BASED NEURAL NETWORKS

Original Abstract Submitted

methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing images using self-attention based neural networks. one of the methods includes obtaining one or more images comprising a plurality of pixels; determining, for each image of the one or more images, a plurality of image patches of the image, wherein each image patch comprises a different subset of the pixels of the image; processing, for each image of the one or more images, the corresponding plurality of image patches to generate an input sequence comprising a respective input element at each of a plurality of input positions, wherein a plurality of the input elements correspond to respective different image patches; and processing the input sequences using a neural network to generate a network output that characterizes the one or more images, wherein the neural network comprises one or more self-attention neural network layers.