18007867. SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO simplified abstract (NVIDIA Corporation)
Contents
SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO
Organization Name
Inventor(s)
Evgeny Aleksandrovich Tumanov of Moscow (RU)
Dmitry Aleksandrovich Korobchenko of Moscow (RU)
Simon Yuen of Playa Vista CA (US)
Kevin Margo of Los Gatos CA (US)
SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO - A simplified explanation of the abstract
This abstract first appeared for US patent application 18007867 titled 'SYNTHETIC AUDIO-DRIVEN BODY ANIMATION USING VOICE TEMPO
Simplified Explanation: The patent application describes a method for generating animations using audio-driven body animation synthesized with voice tempo. This involves using audio input of recorded speech to drive full body animations, with voice tempo used to compare the audio signal to datasets containing animations and corresponding audio signals. Transition points between animations are identified and stitched together seamlessly.
Key Features and Innovation:
- Generation of animations using audio-driven body animation synthesized with voice tempo
- Comparison of audio signals from input audio to datasets for optimal transition points
- Stitching together animations using interpolation and neural networks
Potential Applications: This technology could be used in:
- Entertainment industry for creating realistic animations
- Virtual reality and augmented reality applications
- Language learning tools for visualizing speech patterns
Problems Solved:
- Efficient generation of animations from audio input
- Seamless stitching of animations for smooth transitions
- Enhanced user experience in interactive applications
Benefits:
- Realistic and synchronized animations with audio
- Improved storytelling in multimedia content
- Enhanced user engagement and immersion
Commercial Applications: The technology could be utilized in:
- Animation studios for faster production processes
- Gaming industry for realistic character animations
- Educational software for interactive learning experiences
Prior Art: Prior research in the field of audio-driven animation synthesis and motion capture technologies could provide insights into similar approaches and techniques.
Frequently Updated Research: Stay updated on advancements in audio-driven animation synthesis, neural network applications in animation, and real-time motion capture technologies.
Questions about Audio-Driven Body Animation Synthesis: 1. How does voice tempo play a role in generating animations from audio input? 2. What are the potential challenges in seamlessly stitching together animations based on audio signals?
Original Abstract Submitted
In various examples, animations may be generated using audio-driven body animation synthesized with voice tempo. For example, full body animation may be driven from an audio input representative of recorded speech, where voice tempo (e.g., a number of phonemes per unit time) may be used to generate a 1D audio signal for comparing to datasets including data samples that each include an animation and a corresponding 1D audio signal. One or more loss functions may be used to compare the 1D audio signal from the input audio to the audio signals of the datasets, as well as to compare joint information of joints of an actor between animations of two or more data samples, in order to identify optimal transition points between the animations. The animations may then be stitched together—e.g., using interpolation and/or a neural network trained to seamlessly stitch sequences together—using the transition points.
