17658706. Reference-Based Super-Resolution for Image and Video Enhancement simplified abstract (Apple Inc.)
Reference-Based Super-Resolution for Image and Video Enhancement
Organization Name
Inventor(s)
Reference-Based Super-Resolution for Image and Video Enhancement - A simplified explanation of the abstract
This abstract first appeared for US patent application 17658706 titled 'Reference-Based Super-Resolution for Image and Video Enhancement
Simplified Explanation
The patent application describes a technology for enhancing images in multi-camera systems by using images captured by cameras with different optical properties and sensors.
- The technology uses a neural network to produce enhanced output images that match the quality characteristics of each region in a lower quality input image with the corresponding region in a higher quality input image.
- The neural network employs robust feature aggregation techniques to blend between single-image enhancement and feature aggregation, based on the specific requirements of each region.
- The guidance regions from the higher quality image are determined for each region in the lower quality image using homographic mapping and semantic feature matching techniques.
Potential Applications
- Multi-camera systems in smartphones or other mobile devices to improve image quality.
- Surveillance systems that utilize multiple cameras to capture high-quality images of a scene.
- Medical imaging systems that combine images from different sensors to enhance diagnostic accuracy.
Problems Solved
- Overcoming the limitations of individual cameras by leveraging the strengths of multiple cameras with different optical properties and sensors.
- Enhancing image quality in regions of lower quality images by referencing higher quality images.
- Providing a more comprehensive and detailed view of a scene by combining images from different cameras.
Benefits
- Improved image quality and resolution in multi-camera systems.
- Enhanced ability to capture fine details and accurately represent different regions of a scene.
- Increased flexibility in adjusting the level of enhancement based on the specific requirements of each region.
Original Abstract Submitted
Devices, methods, and computer readable media to provide enhanced images in multi-camera systems, e.g., by using images captured by cameras with different optical properties and/or sensors. In one embodiment, the techniques comprise reference-based image super-resolution techniques for producing, with a first neural network employing robust feature aggregation techniques (e.g., techniques able to blend between single-image enhancement and feature aggregation, when appropriate), an enhanced output image that attempts to match the quality characteristics of each of region in a lower quality (e.g., shorter focal length, larger field of view (FOV)) input image with the quality characteristics of the region's determined guidance region from at least a second, i.e., higher quality (e.g., longer focal length, smaller FOV image) input image. The guidance regions from the higher quality image that are determined for each region from the lower quality input image may be determined by performing homographic mapping and/or semantic feature matching techniques.
