View source for US Patent Application 18355243. Systems and Methods for Machine-Learned Models Having Convolution and Attention simplified abstract

=Systems and Methods for Machine-Learned Models Having Convolution and Attention=


==Organization Name==

[[Google LLC]]


==Inventor(s)==

[[Zihang Dai of Cupertino CA (US)]]

[[Mingxing Tan of Newark CA (US)]]

[[Quoc V. Le of Sunnyvale CA (US)]]

[[Hanxiao Liu of Santa Clara CA (US)]]

==Systems and Methods for Machine-Learned Models Having Convolution and Attention - A simplified explanation of the abstract==


This abstract first appeared for US patent application 18355243 titled 'Systems and Methods for Machine-Learned Models Having Convolution and Attention

==Simplified Explanation==

The patent application describes a method for computer vision that reduces computational cost and improves accuracy.
* The method involves using a machine-learned convolutional attention network to process input data.
* The convolutional attention network consists of multiple stages and includes at least one attention block.
* The attention block uses a relative attention mechanism, which combines a static convolution kernel with an adaptive attention matrix.
* This approach improves the generalization, capacity, and efficiency of the convolutional attention network compared to existing models.


==Original Abstract Submitted==

A computer-implemented method for performing computer vision with reduced computational cost and improved accuracy can include obtaining, by a computing system including one or more computing devices, input data comprising an input tensor having one or more dimensions, providing, by the computing system, the input data to a machine-learned convolutional attention network, the machine-learned convolutional attention network including two or more network stages, and, in response to providing the input data to the machine-learned convolutional attention network, receiving, by the computing system, a machine-learning prediction from the machine-learned convolutional attention network. The convolutional attention network can include at least one attention block, wherein the attention block includes a relative attention mechanism, the relative attention mechanism including the sum of a static convolution kernel with an adaptive attention matrix. This provides for improved generalization, capacity, and efficiency of the convolutional attention network relative to some existing models.