Gopro, Inc.

Ryan Lustig

Balineedu Chowdary Adsumilli

David A Newman

Aaron Staranowicz

Desmond Chik

Daryl Stimm

Joven Matias

Kyler William Schwartz

Joseph VanDalsem

A processing device generates composite images from a sequence of images. The composite images may be used as frames of video. A foreground/background segmentation is performed at selected frames to extract a plurality of foreground object images depicting a foreground object at different locations as it moves across a scene. The foreground object images are stored to a foreground object list. The foreground object images in the foreground object list are overlaid onto subsequent video frames that follow the respective frames from which they were extracted, thereby generating a composite video.

Automatic composition of composite images or videos from frames captured with moving camera

A target image captured from a fisheye lens or other lens with known distortion parameters may be transformed to align it to a reference image. Corresponding features may be detected in the target image and the reference image. The features may be transformed to a spherical coordinate space. In the spherical space, images may be re-pointed or rotated in three dimensions to align all or a subset of the features of the target image to the corresponding features of the reference image. For example, in a sequence of images, background features of the target image in the spherical image space may be aligned to background features of the reference image in the spherical image space to compensate for camera motion while preserving foreground motion. An inverse transformation may then be applied to bring the images back into the original image space.

Image alignment using a virtual gyroscope model

Methods and apparatus for the generation of interpolated frames of video data. In one embodiment, the interpolated frames of video data are generated by obtaining two or more frames of video data from a video sequence; determining frame errors for the obtained two or more frames from the video sequence, determining whether the frame errors exceed a threshold value; performing a multi-pass operation; performing a single-pass operation; performing frame blending; performing edge correction; and generating the interpolated frame of image data.

Apparatus and methods for artifact detection and removal using frame interpolation techniques

An algorithm for performing an image or video processing task is generated that may be used to combine a plurality of different independent solutions to the image or video processing task in an optimized manner. A plurality of base algorithms may be applied to a training set of images or video and a first generation of different combining algorithms may be applied to combine the respective solutions from each of the respective base algorithms into respective combined solutions. The respective combined solutions may be evaluated to generate respective fitness scores representing measures of how well the plurality of different combining algorithms each perform the image or video processing task. The algorithms may be iteratively updated to generate an optimized combining algorithm that may be applied to an input image or video.

Combining independent solutions to an image or video processing task

Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. In one example, best scenes are identified based on the motion values associated with frames or portions of a frame of a video. Motion values are determined for each frame and portions of the video including frames with the most motion are identified as best scenes. Best scenes may also be identified based on the motion profile of a video. The motion profile of a video is a measure of global or local motion within frames throughout the video. For example, best scenes are identified from portion of the video including steady global motion. A video summary can be generated including one or more of the identified best scenes.

Scene and activity identification in video summary generation based on motion detected in a video

Apparatus and methods for video image post-processing for segmentation based interpolation. In one embodiment, a computerized apparatus is utilized in order to obtain a first frame of video data; segment one or more objects within the first frame of video data; obtain a second frame of video data; segment one or more objects within the second frame of video data; match at least a portion of the one or more objects within the first frame of video data with the one or more objects within the second frame of video data; compute the motion of the pixels for the matched portion of the one or more objects; compute the motion of pixels associated with a background image; and generate an interpolated frame of video data, the interpolated frame of video data residing temporally between the first frame of video data and the second frame of video data.

Apparatus and methods for video image post-processing for segmentation-based interpolation

Systems and methods for image compositing. Image capture devices may be used for documenting various activities. An image capture device may be configured to operate in image acquisition mode configured for image compositing. The image acquisition mode may be characterized by one or more of a given image acquisition interval, number of acquired images, camera shutter setting, and/or other settings. The image capture device may provide a user interface configured to facilitate acquiring an image sequence suitable for compositing. A live compositing preview may enable users to select one or more composite images for viewing and/or sharing.

User interface for creating composite images

Methods and apparatus for the generation of interpolated frames of video data. In one embodiment, the interpolated frames of video data are generated by obtaining two or more frames of video data; performing Lagrangian interpolation on one or more portions of the obtained two or more frames of video data to generate a Lagrangian interpolated image; performing Eulerian interpolation on one or more portions of the obtained two or more frames to generate a Eulerian interpolated image; and when the Lagrangian interpolated image and the Eulerian interpolated image should be combined, computing an average interpolated image using the Lagrangian interpolated image and the Eulerian interpolated image; otherwise, selecting either the Lagrangian interpolated image or the Eulerian interpolated image; and generating an interpolated frame of video data using one or more of the average interpolated image, the Lagrangian interpolated image, or the Eulerian interpolated image.

Growing community of inventors

Encinitas, CA, United States of America

Ryan Lustig