18053027. HARMONIZING COMPOSITE IMAGES UTILIZING A SEMANTIC-GUIDED TRANSFORMER NEURAL NETWORK simplified abstract (ADOBE INC.)

HARMONIZING COMPOSITE IMAGES UTILIZING A SEMANTIC-GUIDED TRANSFORMER NEURAL NETWORK

Organization Name

ADOBE INC.

Inventor(s)

He Zhang of San Jose CA (US)

Hyun Joon Jung of Monte Sereno CA (US)

HARMONIZING COMPOSITE IMAGES UTILIZING A SEMANTIC-GUIDED TRANSFORMER NEURAL NETWORK - A simplified explanation of the abstract

This abstract first appeared for US patent application 18053027 titled 'HARMONIZING COMPOSITE IMAGES UTILIZING A SEMANTIC-GUIDED TRANSFORMER NEURAL NETWORK

Simplified Explanation

The present disclosure describes a system, computer-readable media, and methods for implementing a multi-branch harmonization neural network architecture to harmonize composite images. The system includes a convolutional branch, a transformer branch, and a semantic branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask.

• Convolutional branch: Extracts localized information from the input composite image using convolutional neural network layers followed by a style normalization layer.
• Transformer branch: Extracts global information based on different resolutions of the input composite image using transformer neural network layers.
• Semantic branch: Generates semantic features through a visual neural network to inform the harmonization of the composite images.

---

1. 1. 1. Potential Applications

- Image editing software - Augmented reality applications - Medical imaging technology

1. 1. 1. Problems Solved

- Enhancing the quality of composite images - Improving the harmonization process in image editing - Streamlining the integration of different neural network branches

1. 1. 1. Benefits

- Enhanced image harmonization capabilities - Efficient extraction of localized and global information - Improved semantic feature generation for composite images

1. 1. 1. Potential Commercial Applications
         1. Optimizing Image Editing Processes with Multi-Branch Harmonization Neural Network Architecture

---

1. 1. 1. Possible Prior Art

There may be prior art related to image harmonization techniques using neural networks, but specific examples are not provided in this disclosure.

---

1. 1. 1. 1. Unanswered Questions

1. 1. 1. How does this system compare to existing image harmonization methods?

This article does not provide a direct comparison with existing image harmonization methods, leaving the reader to wonder about the specific advantages of this multi-branch neural network architecture.

1. 1. 1. What are the computational requirements for implementing this system?

The computational resources needed to run this multi-branch harmonization neural network architecture are not detailed in the abstract, leaving a gap in understanding the practical implications of deploying this technology.

Original Abstract Submitted

The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a multi-branch harmonization neural network architecture to harmonize composite images. For example, in one or more implementations, the semantic-guided transformer-based harmonization system uses a convolutional branch, a transformer branch, and a semantic branch to generate a harmonized composite image based on an input composite image and a corresponding segmentation mask. More particularly, the convolutional branch comprises a series of convolutional neural network layers followed by a style normalization layer to extract localized information from the input composite image. Further, the transformer branch comprises a series of transformer neural network layers to extract global information based on different resolutions of the input composite image. The semantic branch includes a visual neural network that generates semantic features that inform the harmonization of the composite images.