18055301. PREDICTING VIDEO EDITS FROM TEXT-BASED CONVERSATIONS USING NEURAL NETWORKS simplified abstract (ADOBE INC.)

PREDICTING VIDEO EDITS FROM TEXT-BASED CONVERSATIONS USING NEURAL NETWORKS

Organization Name

ADOBE INC.

Inventor(s)

Uttaran Bhattacharya of Sunnyvale CA (US)

Gang Wu of San Jose CA (US)

Viswanathan Swaminathan of Saratoga CA (US)

Stefano Petrangeli of Mountain View CA (US)

PREDICTING VIDEO EDITS FROM TEXT-BASED CONVERSATIONS USING NEURAL NETWORKS - A simplified explanation of the abstract

This abstract first appeared for US patent application 18055301 titled 'PREDICTING VIDEO EDITS FROM TEXT-BASED CONVERSATIONS USING NEURAL NETWORKS

Simplified Explanation

The patent application describes a system that uses neural networks to predict and apply editing operations to a video sequence based on conversational messages processed by a video editing system.

• The system receives input containing a video sequence and text sentences describing modifications to the video.
• A first neural network maps the text descriptions to candidate editing operations.
• A second neural network processes the video sequence to predict parameter values for the candidate editing operations.
• The system generates a modified video sequence by applying the candidate editing operations with the predicted parameter values.

Potential Applications

This technology could be used in video editing software to automate the editing process based on user input, making it easier and more efficient to create edited videos.

Problems Solved

This technology solves the problem of manually editing videos by automating the process based on natural language descriptions, saving time and effort for video editors.

Benefits

The benefits of this technology include increased efficiency in video editing, improved accuracy in applying editing operations, and a more user-friendly editing experience.

Potential Commercial Applications

This technology could be applied in video editing software for professionals, content creators, and social media influencers looking to streamline their editing process and produce high-quality videos more quickly.

Possible Prior Art

One possible prior art could be the use of machine learning algorithms in video editing software to assist users in applying editing effects and transitions automatically based on user input.

Unanswered Questions

How does the system handle complex editing operations that may require multiple steps or parameters?

The patent application does not provide details on how the system handles complex editing operations that may involve multiple parameters or steps.

What is the accuracy rate of the neural networks in predicting editing operations and parameter values?

The patent application does not mention the accuracy rate of the neural networks in predicting editing operations and parameter values, which could be crucial for assessing the reliability of the system.

Original Abstract Submitted

Embodiments are disclosed for predicting, using neural networks, editing operations for application to a video sequence based on processing conversational messages by a video editing system. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including a video sequence and text sentences, the text sentences describing a modification to the video sequence, mapping, by a first neural network content of the text sentences describing the modification to the video sequence to a candidate editing operation, processing, by a second neural network, the video sequence to predict parameter values for the candidate editing operation, and generating a modified video sequence by applying the candidate editing operation with the predicted parameter values to the video sequence.