Apple Inc.

Jason Klivington

Rudolph Van Der Merwe

Arwen V Bradley

Richard E Crandall

Douglas P Mitchell

Samuel Noble

Sebastien X Beysserie

Charles A Mezak

Piotr J Stanczyk

Evan T Jones

Mark Zimmer

Jianping Zhou

George E Williams

Etienne H Guerard

Damien J Thivent

Xiaoxing Li

Anders M Holtsberg

Joseph M Triscari

Martin Lindberg

Duncan Robert Kerr

Yang Yang

Behkish J Manzari

Paul M Hubel

Farhan A Baqai

Matan Stauber

Vignesh Jagadeesh

Todd Stephen Sachs

Frank Doepke

Rolf Toft

Claus Molgaard

Amir Hoffnung

Jacques P Gasselin De Richebourg

Alex Tremain Nelson

Mitchell Howard Oslick

Emilie Kim

Afshin Dehghan

Paul Thomas Schneider

Claus Moelgaard

David Anthony Kramer

Shuang Gao

Robert A Bailey

Zahra Sadeghipoor Kermani

Vasilios E Anton

Sebastien Marineau-Mes

Ran Margolin

Nikhil P Singh

Joe Triscari

Sébastien Beysserie

Techniques are disclosed for editing captured media to overcome operational difficulties that may arise during capture operations. According to these techniques, content may be captured with a pair of cameras, a first camera having a wider field of view than a second camera. Object(s) may be detected from captured content from the wider field of view camera. The captured content may be processed from the wider field of view camera in a location of at least one detected object. Typically, operators may attempt to frame content using content from the narrower field of view camera. As a result, operators may be unaware that desired content is captured using a second, wider field of view camera. Results from the processed wider field of view data may be proposed to operators for review and, if desired, retention.

Video analysis and management techniques for media capture and retention

Techniques and devices for generating multiple output video variations for an input video based on a shared resource architecture. The shared resource architecture reuses and shares computational and gating results from one or more operations to create the multiple output video variations. The shared resource architecture applies a frame-time normalization of the trimmed and stabilized video to produce a trimmed stabilized normalized video and, thereafter, uses the trimmed stabilized normalized video to precompute one or more video parameters that can be shared with multiple output video variations. The shared resource architecture can then generate multiple output video variations using the shared video parameters.

Seamless output video variations for an input video

In various implementations a method includes obtaining a plurality of source images, stabilizing the plurality of source images to generate a plurality of stabilized images, and averaging the plurality of stabilized image to generate a synthetic long exposure image. In various implementations, stabilizing the plurality of source images includes: selecting one of the plurality of source images to serve as a reference frame; and registering others of the plurality of source images to the reference frame by applying a perspective transformation to others of the plurality of the source images.

Synthetic long exposure image with optional enhancement using a guide image

Techniques and devices for creating a Forward-Reverse Loop output video and other output video variations. A pipeline may include obtaining input video and determining a start frame within the input video and a frame length parameter based on a temporal discontinuity minimization. The selected start frame and the frame length parameter may provide a reversal point within the Forward-Reverse Loop output video. The Forward-Reverse Loop output video may include a forward segment that begins at the start frame and ends at the reversal point and a reverse segment that starts after the reversal point and plays back one or more frames in the forward segment in a reverse order. The pipeline for the generating Forward-Reverse Loop output video may be part of a shared resource architecture that generates other types of output video variations, such as AutoLoop output videos and Long Exposure output videos.

Seamless forward-reverse video loops

Techniques and devices for creating an AutoLoop output video include performing postgate operations. The AutoLoop output video is created from a set of frames. After generating the AutoLoop output video based on a plurality of loop parameters and at least a portion of the frames, postgate operations determine one or more dynamism metrics based on a variability metric and a dynamic range metric for a plurality of pixels within the video loop. Postgate operations compare the dynamism metrics to one or more postgate threshold values and reject the video loop based on the comparison of the dynamism metrics to the postgate threshold values.

Automated seamless video loop

Techniques and devices for creating an AutoLoop output video include identifying optimal loops within short videos or within a series of image. The AutoLoop output video may be automatically created using casually shot, handheld videos, and may include an AutoLoop pipeline that may comprise obtaining an input video, stabilizing the input video, detecting optimal loop parameters and baking out the AutoLoop output video with crossfade and playing back the AutoLoop output video. Video stabilization can include a cascade of video stabilization algorithms including a tripod-direct mode and a tripod-sequential mode. After stabilization, an AutoLoop operation may determine optimal loop parameters. Once optimal loop parameters are determined, a crossfade may be added to smooth out any temporal and spatial discontinuities in the AutoLoop output video.

Techniques and devices for creating an AutoLoop output video include performing pregate operations. The AutoLoop output video is created from a set of frames. Prior to creating the AutoLoop output video, the set of frames are automatically analyzed to identify one or more image features that are indicative of whether the image content in the set of frames is compatible with creating a video loop. Pregate operations assign one or more pregate scores for the set of frames based on the one or more identified image features, where the pregate scores indicate a compatibility to create the video loop based on the identified image features. Pregate operations automatically determine to create the video loop based on the pregate scores and generate an output video loop based on the loop parameters and at least a portion of the set of frames.

Techniques and devices for creating an AutoLoop output video by adding synthetic camera motion to the AutoLoop output video. The AutoLoop output video is created from a set of frames. After generating the AutoLoop output video based on a plurality of loop parameters and at least a portion of the frames, synthetic camera motion is combined with the AutoLoop output video. The synthetic camera loop is based on the subset of the input frames and exhibits some amount of camera motion for the subset of the input frames. Once the synthetic camera loop is generated, the synthetic camera loop and the video loop is combined to enhance the AutoLoop output video.

Techniques to permit a digital image capture device to stabilize a video stream in real-time (during video capture operations) are presented. In general, techniques are disclosed for stabilizing video images using an overscan region and a look-ahead technique enabled by buffering a number of video input frames before generating a first stabilized video output frame. (Capturing a larger image than is displayed creates a buffer of pixels around the edge of an image; overscan is the term given to this buffer of pixels.) More particularly, techniques are disclosed for buffering an initial number of input frames so that a 'current' frame can use motion data from both 'past' and &#x201c;future&#x201d; frames to adjust the strength of a stabilization metric value so as to keep the current frame within its overscan. This look-ahead and look-behind capability permits a smoother stabilizing regime with fewer abrupt adjustments.

Video image stabilization

A method of removing unwanted camera motion from a video sequence is provided. The method matches a group of feature points between each pair of consecutive video frames in the video sequence. The method calculates the motion of each matched feature point between the corresponding pair of consecutive video frames. The method calculates a set of historical metrics for each feature point. The method, for each pair of consecutive video frames, identifies a homography that defines a dominant motion between the pair of consecutive frames. The homography is identified by performing a geometrically biased historically weighted RANSAC on the calculated motion of the feature points. The geometrically biased historically weighted RANSAC gives a weight to the calculated motion of each feature point based on the historical metrics calculated for the feature point. The method removes the unwanted camera motion from the video sequence by using the identified homographies.

Robust image feature based video stabilization and smoothing

Systems, methods and computer program products are disclosed for resampling a digital image. According to an implementation, a source image can be presharpened and upsampled to a first upsampled image having a specified image size and a first level of presharpening. The source image is also presharpened and upsampled to a second upsampled image having the specified image size and second level of presharpening that is less than the first level of presharpening. The first and second upsampled images are deblurred. A binary edge mask image is generated from the deblurred, upsampled images. The binary edge mask image is dilated and blurred to generate a deep mask image. The first and second, deblurred upsampled images are blended together using the deep mask image.

Digital image resampling

Traditionally, time-lapse videos are constructed from images captured at time intervals called 'temporal points of interests' or 'temporal POIs.' Disclosed herein are systems and methods of constructing improved, motion-stabilized time-lapse videos using temporal points of interest and image similarity comparisons. According to some embodiments, a &#x201c;burst&#x201d; of images may be captured, centered around the aforementioned temporal points of interest. Then, each burst sequence of images may be analyzed, e.g., by performing an image similarity comparison between each image in the burst sequence and the image selected at the previous temporal point of interest. Selecting the image from a given burst that is most similar to the previous selected image, while minimizing the amount of motion with the previous selected image, allows the system to improve the quality of the resultant time-lapse video by discarding &#x201c;outlier&#x201d; or other undesirable images captured in the burst sequence and motion stabilizing the selected image.

Time-lapse video capture with optimal image stabilization

A method of removing unwanted camera motion from a video sequence is provided. The method matches a group of feature points between each pair of consecutive video frames in the video sequence. The method calculates the motion of each matched feature point between the corresponding pair of consecutive video frames. The method calculates a set of historical metrics for each feature point. The method, for each pair of consecutive video frames, identifies a homography that defines a dominant motion between the pair of consecutive frames. The homography is identified by performing a geometrically biased historically weighted RANSAC on the calculated motion of the feature points. The geometrically biased historically weighted RANSAC gives a weight to the calculated motion of each feature point based on the historical metrics calculated for the feature point. The method removes the unwanted camera motion from the video sequence using the identified homographies.

Systems, methods and computer program products are disclosed for automatic image sharpening. Automatic image sharpening techniques are disclosed that automatically bring a blurred image into focus. Techniques for reducing edge ringing in sharpened images are also disclosed. According to implementations, a computer-implemented method includes determining a normalized entropy of a first image, calculating a correlation target based on the normalized entropy, automatically determining a blur radius of a de-convolution kernel that causes a cosine of a first radial power spectrum of the kernel and a second radial power spectrum of a reconstruction of the first image to approximate the correlation target and generating a second image based on the blur radius.

Automatic image sharpening using entropy-based blur radius

Growing community of inventors

Portland, OR, United States of America

Jason Klivington