Google Inc.

Waymo LLC

The Provost Fellows and Scholars of the College of the Holy and Undivided Trinity of Queen Elizabeth, Near Dublin

Damien Kelly

Anil Christopher Kokaram

Andrew Crawford

Hugh Denman

Neil Birkbeck

Balineedu Adsumilli

Xiaoying He

Jia Feng

Sasi Inguva

Ignacio Garcia Dorado

Stuart Bennett

Mohammad Izadi

Joan Lasenby

Molly Castle Nix

Nirav Dharia

Peyman Milanfar

Jason Matthew Toff

John A Gregg

Rushabh Ashok Doshi

Steve Benting

Benjamin Frevert

Joshua Abraham Tabak

Bartlomiej Maciej Wronski

Christopher Rhodes

Jeremy Doig

Neil Aylon Charles Birkbeck

Perry Tobin

Isasi Inguva

Ka-kit Lam

Debargha Mukherjee

Lucian Ion

Reed Morse

Janne Matias Kontkanen

Isaac William Reynolds

Reed G A Morse

Alexander Schiffhauer

Frank Boland

Kiran Murthy

Ignacio Garcia-Dorado

Francois Pitie

Jiangyang Zhang

Bartlomiej Wronski

Shambhavi Punja

Balineedu C Adsumilli

Andrew Joseph Crawford

Hugh Pierre Denman

An example embodiment may involve capturing a sequence of images, wherein there are 4 or more images in the sequence of images, and wherein each of the sequence of images has an exposure length of 4-100 seconds; applying a sliding window over the sequence of images as downsampled, wherein at least 4 images are encompassed within the sliding window', and wherein for each position of the sliding window the applying involves: (i) aligning a set of images within the sliding window, and (ii) merging the set of images as aligned into a video frame; combining video frames generated by way of the sliding window into a video file; and storing, by the mobile device, the video file in memory of the mobile device.

Mobile device support for capture and synthesis of extreme low-light video

A video item that is subject to one or more motion stabilization transformations applied to the video item is identified. The one or more motion stabilization transformations pertain to a motion between video frames of a video sequence associated with the video item. One or more operations to modify the one or more motion stabilization transformations applied to the video item are determined. The one or more operations to modify the motion stabilization transformations are performed. The video item is provided for playback via a client device. Playback of the video item depicts at least a portion of the motion between the video frames of the video sequence.

Methods and systems for non-destructive stabilization-based encoder optimization

A media items to be shared with users of a content sharing service are identified. Each of the media items corresponds to a video recording generated by a client device that depicts one or more objects corresponding to a real-world event and/or a geographic location. A location of the client device that generated the video recording corresponding to a respective media item of the media items is determined based on image features depicted in a set of frames of the video recording. A request for content associated with at least one of the real-world event and/or the geographic location is received from another client device connected to the content sharing service. The media items and, for each of the media items, an indication of the location of the client device that generated the corresponding video recording are provided in accordance with the request for content.

Determining multiple camera positions from multiple videos

Described examples relate to an apparatus comprising a memory for storing a sequence of image frames and at least one processor. The at least one processor may be configured to receive a first image frame of a sequence of image frames from an image capture device and select a first portion of a first image frame. The at least one processor may also be configured to obtain alignment information and determine a first portion and a second portion of a second image frame based on the alignment information. Further, the at least one processor may be configured to determine a bounding region within the second image frame and fetch image data corresponding to the bounding region of the second image frame from memory. In some examples, the first image frame may comprise a base image and the second image frame may comprise an alternative image frame. Further, the first image frame may comprise any one of the image frames of the sequence of image frames.

Systems, apparatus, and methods for retrieving image data of image frames

Described examples relate to an apparatus comprising a first sensor configured to scan an area of interest during a first time period and a second sensor configured to capture a plurality of images of a field of view. The apparatus may include at least one controller configured to receive the plurality of images captured by the second sensor, compare the timestamp information associated with at least one image of the plurality of images to at least one time period of the first time period, and select a base image from the plurality of images based on the comparison.

Systems, apparatus, and methods for generating enhanced images

Described examples relate to an apparatus comprising one or more image sensors coupled to a vehicle and at least one processor. The at least one processor may be configured to capture, in a burst sequence using the one or more image sensors, multiple frames of an image of a scene, the multiple frames having respective, relative offsets of the image across the multiple frames and perform super-resolution computations using the captured, multiple frames of the image of the scene. The at least one processor may also be configured to accumulate, based on the super-resolution computations, color planes and combine, using the one or more processors, the accumulated color planes to create a super-resolution image of the scene.

Systems, apparatus, and methods for enhanced image capture

Methods and systems for non-destructive, stabilization-based encoder optimization. A video item to be provided to one or more users of a platform is identified. An indication of a motion is identified between an initial video frame of a video sequence associated with the video item and a subsequent video frame of the video sequence. One or more motion stabilization transformations are applied to the video item to modify the motion between at least the initial frame and the subsequent frame. Upon applying the one or more motion stabilization transformations to the video item, the video item is encoded. The encoded video item and one or more instructions to cause the client device to reverse the one or more motion stabilization transformations applied to the video item after decoding the encoded video item are transmitted to a client device connected to the platform.

Processing a spherical video using denoising is described. Video content comprising the spherical video is received. Whether a camera geometry or a map projection, or both, used to generate the spherical video is available is then determined. The spherical video is denoised using a first technique responsive to a determination that the camera geometry, the map projection, or both is available. Otherwise, the spherical video is denoised using a second technique. At least some steps of the second technique can be different from steps of the first technique. The denoised spherical video can be encoded for transmission or storage using less data than encoding the spherical video without denoising.

Method for denoising omnidirectional videos and rectified videos

Signals of an immersive multimedia item are jointly considered for optimizing the quality of experience for the immersive multimedia item. During encoding, portions of available bitrate are allocated to the signals (e.g., a video signal and an audio signal) according to the overall contribution of those signals to the immersive experience for the immersive multimedia item. For example, in the spatial dimension, multimedia signals are processed to determine spatial regions of the immersive multimedia item to render using greater bitrate allocations, such as based on locations of audio content of interest, video content of interest, or both. In another example, in the temporal dimension, multimedia signals are processed in time intervals to adjust allocations of bitrate between the signals based on the relative importance of such signals during those time intervals. Other techniques for bitrate optimizations for immersive multimedia streaming are also described herein.

Bitrate optimizations for immersive multimedia streaming

An interactive multi-view module identifies a plurality of media items associated with one or more real-world events. Each of the plurality of media items is created by capturing the real-world events from a particular geographic location. The interactive multi-view module determines a geographic position associated with each of the media items and presents the media items in an interactive multi-view player interface based at least on the geographic positions. The interactive multi-view player interface allows a user to switch between of media items, and indicates at least one of a video density indicating a number of media items available at a given point in time or an event highlight indicating a popularity of the media items at a given point in time. The popularity of the respective media items is determined using one or more factors comprising a number of views of the media items at a given point in time.

Multi-view audio and video interactive playback

A set of media items to be shared with users of a content sharing service is identified. Each of the set of media items corresponds to a video recording generated by a client device that depicts one or more objects corresponding to a real-world event at a geographic location. A positioning of the client device that generated the video recording corresponding to a respective media item of the set of media items is determined. The positioning is determined based on image features depicted in a set of frames of the video recording. A request for content associated with at least one of the real-world event or the geographic location is received from another client device connected to the content sharing service. The set of media items and, for each of the set of media items, an indication of the determined positioning of the client device that generated the corresponding video recording is provided in accordance with the request for content.

A method for denoising video content includes identifying a first frame block associated with a first frame of the video content. The method also includes estimating a first noise model that represents characteristics of the first frame block. The method also includes identifying at least one frame block adjacent to the first frame block. The method also includes generating a second noise model that represents characteristics of the at least one frame block adjacent to the first frame block by adjusting the first noise model based on at least one characteristic of the at least one frame block adjacent to the first frame block. The method also includes denoising the at least one frame block adjacent to the first frame block using the second noise model.

Method and apparatus for denoising video content

Growing community of inventors

Sunnyvale, CA, United States of America

Damien Kelly