18486534. Highly Efficient Convolutional Neural Networks simplified abstract (GOOGLE LLC)

Highly Efficient Convolutional Neural Networks

Organization Name

GOOGLE LLC

Inventor(s)

Andrew Gerald Howard of Culver City CA (US)

Mark Sandler of Mountain View CA (US)

Liang-Chieh Chen of Los Angeles CA (US)

Andrey Zhmoginov of Mountain View CA (US)

Menglong Zhu of Playa Vista CA (US)

Highly Efficient Convolutional Neural Networks - A simplified explanation of the abstract

This abstract first appeared for US patent application 18486534 titled 'Highly Efficient Convolutional Neural Networks

Simplified Explanation

The present disclosure introduces new, more efficient neural network architectures that include linear bottleneck layers positioned before and/or after convolutional layers, such as depthwise separable convolutional layers, and inverted residual blocks with thin bottleneck layers as input and output, and expanded representations in between.

• Linear bottleneck layers positioned before and/or after convolutional layers
• Inverted residual blocks with thin bottleneck layers as input and output
• Expanded representations in between, including convolutional layers
• Residual shortcut connections between thin bottleneck layers

Potential Applications

The technology can be applied in various fields such as image recognition, natural language processing, and autonomous driving systems.

Problems Solved

The neural network architectures address the challenge of improving efficiency and performance in deep learning models.

Benefits

The new architectures offer increased efficiency, better performance, and potential for faster training and inference times.

Potential Commercial Applications

This technology can be utilized in industries such as healthcare, finance, and retail for tasks like medical image analysis, fraud detection, and customer behavior prediction.

Possible Prior Art

Prior art may include existing neural network architectures with bottleneck layers and residual connections, but the specific combination of linear bottleneck layers, inverted residual blocks, and expanded representations is novel.

How does this technology compare to existing neural network architectures in terms of computational efficiency and model performance?

The article does not provide a direct comparison with existing architectures to evaluate the improvements in computational efficiency and model performance.

What are the potential limitations or drawbacks of implementing these new neural network architectures in practical applications?

The article does not discuss any potential limitations or drawbacks that may arise when implementing these architectures in real-world scenarios.

Original Abstract Submitted

The present disclosure provides directed to new, more efficient neural network architectures. As one example, in some implementations, the neural network architectures of the present disclosure can include a linear bottleneck layer positioned structurally prior to and/or after one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. As another example, in some implementations, the neural network architectures of the present disclosure can include one or more inverted residual blocks where the input and output of the inverted residual block are thin bottleneck layers, while an intermediate layer is an expanded representation. For example, the expanded representation can include one or more convolutional layers, such as, for example, one or more depthwise separable convolutional layers. A residual shortcut connection can exist between the thin bottleneck layers that play a role of an input and output of the inverted residual block.