Gopro, Inc.

Andrew Russell

Robert McIntosh

Daryl Stimm

Kyler William Schwartz

Robert Mcintosh

Methods and apparatus for post-processing in-camera stabilized video. Embodiments of the present disclosure reconstruct and re-stabilize an in-camera stabilized video to provide for improved stabilization (e.g., a wider crop, etc.). In-camera sensor data may be stored and used to re-calculate orientation metadata in post-production. In-camera stabilization provides several benefits (e.g., the ability to share stabilized videos from the camera without additional post-processing as well as reduced file sizes of the shared videos). Camera-aware post-processing can reuse portions of the in-camera stabilized videos while providing additional benefits (e.g., the ability to regenerate the original captured videos in post-production and re-stabilize the videos). Camera-aware post-processing can also improve orientation metadata and remove sensor error. The disclosed techniques also enable assisted optical flow-based stabilization using the refined metadata.

Methods and apparatus for re-stabilizing video in post-processing

Framings of a video determines which spatial extents of the video are presented during playback of the video. A viewing path for the video is generated based on the framings of the video, and the number of framings for the video is reduced based on the viewing path for the video. The reduced number of framings for the video is provided for use in generating a presentation of the video.

Reducing number of video framing

An image capture device may capture a video while experiencing motion. Motion of the image capture device during video capture may be determined based on optical flow and structure from motion solve of the video. Optical flow derived motion and/or the motion solve derived motion of the image capture device may be selected to perform stabilization of the video.

Systems and methods for stabilizing videos using optical flow derived motion and motion solve derived motion

Systems, apparatus, and methods for piggyback camera calibration. Existing piggybacked capture techniques use a 'beauty camera' and an “action camera” to capture raw footage. The user directly applies the EIS stabilization track of a piggybacked action camera to the cinematic footage to create desired stable footage. Unfortunately, since the action camera may have been slightly offset from the cinematic video camera, the EIS stabilization data will only roughly approximate the necessary corrections. In other words, the user must manually fine tune the corrections. The disclosed embodiments use a calibration sequence to estimate a physical offset between the beauty camera and the action camera. Then, the estimated physical offset can be used to calculate an offset camera orientation for stabilizing the beauty camera. The foregoing process can be performed in-the-field before actual capture. This allows the user to check their set-up and fix any issues before capturing the desired footage.

Systems, methods, and apparatus for piggyback camera calibration

Methods and apparatus for stabilizing image data based on a lens polynomial. Non-rectilinear footage can be captured and rectified in-camera; the rectified images may be stabilized to provide rectified stable video. In one exemplary embodiment, the footage is rectified and stabilized based on a lens polynomial and the camera's own movement. In some variants, the rectified stable video may be stored along with its margin track. In-camera rectified stable video provides several benefits over traditional techniques (e.g., the ability to share rectilinear content from the camera without additional post-processing, as well as reduced file sizes of the shared videos). Lens-aware post-processing can reuse portions of the in-camera rectified stable videos while providing additional benefits (e.g., the ability to re-frame the video in post-production).

Methods and apparatus for electronic image stabilization based on a lens polynomial

Methods and apparatus for post-processing in-camera stabilized video. Embodiments of the present disclosure reconstruct and re-stabilize an in-camera stabilized video to provide for improved stabilization (e.g., a wider crop, etc.) In-camera sensor data may be stored and used to re-calculate orientation metadata in post-production. In-camera stabilization provides several benefits (e.g., the ability to share stabilized videos from the camera without additional post-processing as well as reduced file sizes of the shared videos). Camera-aware post-processing can reuse portions of the in-camera stabilized videos while providing additional benefits (e.g., the ability to regenerate the original captured videos in post-production and re-stabilize the videos). Camera-aware post-processing can also improve orientation metadata and remove sensor error. The disclosed techniques also enable assisted optical flow-based stabilization using the refined metadata.

Poses of a person depicted within video frame may be determined. The poses of the person may be used to generate intermediate video frames between the video frames.

Pose estimation for frame interpolation

An image capture device may capture video frames while experiencing motion. The video frames may be stored in a visual track, and rotational positions of the image capture device may be stored in a rotational position track. The visual track and the rotational position track may not be temporally aligned. A looping stabilized view of the view frames and a synchronization adjustment option may be presented on a display. The synchronization adjustment option may enable user modification of timing of the rotational positions stored within the rotational position track. Timing modification of the rotational positions may cause changes in the looping stabilized view, which may be used as feedback on whether the synchronization between the visual track and the rotational position track is being improved or not by the timing modification of the rotational positions.

Growing community of inventors

San Mateo, CA, United States of America

Andrew Russell