NVIDIA Corporation (20240233229). SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO simplified abstract
Contents
SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO
Organization Name
Inventor(s)
Evgeny Aleksandrovich Tumanov of Moscow (RU)
Dmitry Aleksandrovich Korobchenko of Moscow (RU)
Simon Yuen of Playa Vista CA (US)
Kevin Margo of Los Gatos CA (US)
SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240233229 titled 'SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO
Simplified Explanation: The patent application describes a method of generating animations using audio-driven body animation synthesized with voice tempo.
Key Features and Innovation:
- Full body animation driven by audio input representing recorded speech.
- Voice tempo used to generate a 1D audio signal for comparison with datasets containing animations and corresponding audio signals.
- Loss functions used to compare audio signals and joint information of actors' joints between animations to identify optimal transition points.
- Animations stitched together using interpolation and neural networks trained for seamless transitions.
Potential Applications: This technology could be used in the entertainment industry for creating realistic animations in movies, video games, and virtual reality experiences. It could also have applications in virtual assistants and communication tools.
Problems Solved: This technology addresses the challenge of creating natural-looking animations that synchronize with audio inputs, improving the overall quality and realism of animated content.
Benefits:
- Enhanced realism in animations.
- Improved synchronization between audio and visual elements.
- Streamlined animation production process.
Commercial Applications: The technology could be utilized by animation studios, game developers, virtual reality companies, and communication technology firms to enhance their products and services.
Questions about the Technology: 1. How does this technology improve the efficiency of animation production? 2. What are the potential limitations of using voice tempo to drive body animations?
Frequently Updated Research: Researchers are continually exploring advancements in audio-driven animation techniques, including improving the accuracy of voice tempo analysis and enhancing the realism of synthesized animations.
Original Abstract Submitted
in various examples, animations may be generated using audio-driven body animation synthesized with voice tempo. for example, full body animation may be driven from an audio input representative of recorded speech, where voice tempo (e.g., a number of phonemes per unit time) may be used to generate a 1d audio signal for comparing to datasets including data samples that each include an animation and a corresponding 1d audio signal. one or more loss functions may be used to compare the 1d audio signal from the input audio to the audio signals of the datasets, as well as to compare joint information of joints of an actor between animations of two or more data samples, in order to identify optimal transition points between the animations. the animations may then be stitched together—e.g., using interpolation and/or a neural network trained to seamlessly stitch sequences together—using the transition points.
