18151430. TRANSFORMING COMPUTER GAME AUDIO USING IMPULSE RESPONSE OF A VIRTUAL 3D SPACE GENERATED BY NERF INPUT TO A CONVOLUTIONAL REVERBERATION ENGINE simplified abstract (Sony Interactive Entertainment Inc.)

TRANSFORMING COMPUTER GAME AUDIO USING IMPULSE RESPONSE OF A VIRTUAL 3D SPACE GENERATED BY NERF INPUT TO A CONVOLUTIONAL REVERBERATION ENGINE

Organization Name

Sony Interactive Entertainment Inc.

Inventor(s)

Michael Taylor of San Mateo CA (US)

TRANSFORMING COMPUTER GAME AUDIO USING IMPULSE RESPONSE OF A VIRTUAL 3D SPACE GENERATED BY NERF INPUT TO A CONVOLUTIONAL REVERBERATION ENGINE - A simplified explanation of the abstract

This abstract first appeared for US patent application 18151430 titled 'TRANSFORMING COMPUTER GAME AUDIO USING IMPULSE RESPONSE OF A VIRTUAL 3D SPACE GENERATED BY NERF INPUT TO A CONVOLUTIONAL REVERBERATION ENGINE

The abstract describes a patent application for using a 3D neural radiance field (NeRF) to generate an impulse response (IR) characterization, which is then input to a convolutional reverberation engine to create an immersive audio experience on a 2.0 stereo speaker system. The NeRF recreates a background geometry, and the virtual 3D space's impulse response is used to transform game sounds/music to appear as though they are occurring inside the game's virtual space. This technology can also blend the virtual and real-world spaces in real-time, tracking the player as they move around the room to create audio as it would sound if the player were moving through the virtual space.

• NeRF technology used to generate impulse response (IR) characterization
• IR input to convolutional reverberation engine for immersive audio experience
• Transform game sounds/music to appear as though they are occurring inside the virtual space
• Blend virtual and real-world spaces in real-time for a more immersive audio experience
• Track player movement to adjust audio based on real-world acoustics

Potential Applications: - Virtual reality gaming - Augmented reality experiences - Audio production and mixing - Virtual concerts and events - Architectural and acoustic design

Problems Solved: - Enhancing audio immersion in virtual environments - Creating a more realistic audio experience in games and simulations - Blending virtual and real-world audio seamlessly - Tracking player movement for dynamic audio adjustments - Simulating acoustics in different environments

Benefits: - Enhanced audio immersion - Realistic spatial audio effects - Seamless blending of virtual and real-world audio - Dynamic audio adjustments based on player movement - Accurate simulation of acoustics in different environments

Commercial Applications: Title: Immersive Audio Technology for Virtual Environments Description: This technology can be used in virtual reality gaming, augmented reality experiences, audio production, virtual concerts, and architectural design to create realistic and immersive audio environments. The market implications include enhanced user experiences, increased engagement, and new opportunities for audio content creators.

Questions about Immersive Audio Technology: 1. How does this technology improve the audio experience in virtual environments? 2. What are the potential applications of blending virtual and real-world audio in real-time?

Original Abstract Submitted

A 3D neural radiance field (NeRF) is used to generate an impulse response (IR) characterization, which can then be input to a convolutional reverberation engine to create an audio experience that reflects the in-game world on a 2.0 stereo speaker system. The NeRF recreates a background geometry and the impulse response of a virtual 3D space generated using NeRF is input to the convolutional reverberation engine to transform game sounds/music to appear as though they are occurring inside the 3D space of the game. The same may be done for the player's real-world space in which the virtual IR and real IR are blended together in real-time, and the real-life player tracked as he moves around the room to create audio as it would sound were the player moving through the virtual space while adjusting for the acoustics of the real space.