Apple Inc.

Sony Corporation

Sony Electronics Inc

Other

Rolf Toft

Paul M Hubel

Marius Tico

Jianping Zhou

Claus Molgaard

George Edward Williams

Thomas Patrick Dawson

Sebastien X Beysserie

George E Williams

David Boyden

Anita Nariani Schulze

Leo M Pedlow, Jr

Anthony Lionel Creed

Toshiro Ozawa

Aaron John Dew

Matthew Chang

Damien J Thivent

Xudong Yu

Brant L Candelore

Peter Rae Shintani

Robert Hardacker

Christopher Jensen Read

Djung N Nguyen

Richard Lawrence Baer

Robert Winston Blanchard

Behram Mario Dacosta

Yuko Nishikawa

Leo Pedlow

Ludovic Etienne Douillet

Seth Hill

Edgar Allan Tu

Milton Massey Frazier

Takashi Hironaka

Rajesh Rao

Mark Hanson

Greg D Gudorf

Nicholas James Colsey

Ian Charles Matthews

Jason R Meerbergen

Ralph T Brunner

Todd Stephen Sachs

Ted Dunn

Frank Doepke

Wei Sun

Takaaki Ota

Jason Klivington

Russell A Pflughaupt

Leo Mark Pedlow, Jr

Robert L Hardacker

Rajesh K Rao

George Williams

Mark A Hanson

Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a 'spatial difference map.' Spatial difference maps may be used to identify pixels in the short- and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

Scene motion correction in fused image systems

Techniques to capture and fuse short- and long-exposure images of a scene from a stabilized image capture device are disclosed. More particularly, the disclosed techniques use not only individual pixel differences between co-captured short- and long-exposure images, but also the spatial structure of occluded regions in the long-exposure images (e.g., areas of the long-exposure image(s) exhibiting blur due to scene object motion). A novel device used to represent this feature of the long-exposure image is a 'spatial difference map.' Spatial difference maps may be used to identify pixels in the short-and long-exposure images for fusion and, in one embodiment, may be used to identify pixels from the short-exposure image(s) to filter post-fusion so as to reduce visual discontinuities in the output image.

Techniques to improve a digital image capture device's ability to stabilize a video stream are presented. According to some embodiments, improved stabilization of captured video frames is provided by intelligently harnessing the complementary effects of both optical image stabilization (OIS) and electronic image stabilization (EIS). In particular, OIS may be used to remove intra-frame motion blur that is typically lower in amplitude and dominates with longer integration times, while EIS may be used to remove residual unwanted frame-to-frame motion that is typically larger in amplitude. The techniques disclosed herein may also leverage information provided from the image capture device's OIS system to perform improved motion blur-aware video stabilization strength modulation, which permits better video stabilization performance in low light conditions, where integration times tend to be longer, thus leading to a greater amount of motion blurring in the output stabilized video.

Combined optical and electronic image stabilization

Systems, methods, and computer readable media to improve image stabilization operations are described. A novel combination of image quality and commonality metrics are used to identify a reference frame from a set of commonly captured images which, when the set's other images are combined with it, results in a quality stabilized image. The disclosed image quality and commonality metrics may also be used to optimize the use of a limited amount of image buffer memory during image capture sequences that return more images that the memory may accommodate at one time. Image quality and commonality metrics may also be used to effect the combination of multiple relatively long-exposure images which, when combined with a one or more final (relatively) short-exposure images, yields images exhibiting motion-induced blurring in interesting and visually pleasing ways.

Reference frame selection for still image stabilization

Traditionally, time-lapse videos are constructed from images captured at time intervals called 'temporal points of interests' or 'temporal POIs.' Disclosed herein are systems and methods of constructing improved, motion-stabilized time-lapse videos using temporal points of interest and image similarity comparisons. According to some embodiments, a &#x201c;burst&#x201d; of images may be captured, centered around the aforementioned temporal points of interest. Then, each burst sequence of images may be analyzed, e.g., by performing an image similarity comparison between each image in the burst sequence and the image selected at the previous temporal point of interest. Selecting the image from a given burst that is most similar to the previous selected image, while minimizing the amount of motion with the previous selected image, allows the system to improve the quality of the resultant time-lapse video by discarding &#x201c;outlier&#x201d; or other undesirable images captured in the burst sequence and motion stabilizing the selected image.

Time-lapse video capture with optimal image stabilization

Systems, methods, and computer readable media to improve image stabilization operations are described. A novel approach to pixel-based registration of non-reference images to a reference frame in a set of commonly captured images is disclosed which makes use of pyramid decomposition to more efficiently detect corners. The disclosed pixel-based registration operation may also be combined with motion sensor data-based registration approaches to register non-reference images with respect to the reference frame. When the registered non-reference images are combined with the pre-selected reference image, the resulting image is a quality stabilized image.

Image registration methods for still image stabilization

Techniques are disclosed for selectively capturing, retaining, and combining multiple sub-exposure images or brackets to yield a final image having diminished motion-induced blur and good noise characteristics. More specifically, after or during the capture of N brackets, the M best may be identified for combining into a single output image, (N&#x3e;M). As used here, the term 'best' means those brackets that exhibit the least amount of relative motion with respect to one another&#x2014;with one caveat: integer pixel shifts may be preferred over sub-pixel shifts.

Dynamic bracketing operations for image stabilization

In some embodiments, a method for compensating for lens motion includes estimating a starting position of a lens assembly associated with captured pixel data. The captured pixel data is captured from an image sensor. In some embodiments, the method further includes calculating from the starting position and position data received from the one or more position sensors lens movement associated with the captured pixel data. The lens movement is mapped into pixel movement associated with the captured pixel data. A transform matrix is adjusted to reflect at least the pixel movement. A limit factor associated with the position data is calculated. The captured pixel data is recalculated using the transform matrix and the limit factor.

Video rolling shutter correction for lens movement in optical image stabilization cameras

Systems, methods, and computer readable media to improve image stabilization operations and other image processing operations are described. A novel combination of interleaved image capture and image processing operations, e.g., image registration operations, may be employed on a bracketed capture of still images. Such techniques may result in improved camera performance and processing efficiency, as well as decreased shot-to-shot time intervals. In another embodiment, an image fusion portion of an image post-processing pipeline may also be performed in an interleaved fashion, such that each image in the sequence of obtained bracketed images may be incrementally added to an output composite image after it has been aligned with the preceding image or images from the sequence.

Interleaving image processing and image capture operations

Growing community of inventors

Palo Alto, CA, United States of America

Rolf Toft