Gopro, Inc.

Meta Platforms, Inc.

Ingrid Anda Cotoros

Jonathan Stern

Nicholas D Woodman

Ian Griggs

Mark Vaden

Peiqian Zhao

Valérian Dottel

Scott Novell

Guillaume Tilloux

Loïc Lanoir

Loïc Bailliard

Thomas Nicolas Emmanuel Veit

Sylvain Leroy

Heng Zhang

Vikas Chandra

Sandor Lee Barna

Michael Brookmire

Jessica Bonner

Robert George Hoffman

Rakesh Ranjan

Christopher J Clark

Yuchen Fan

Xiaoyu Xiang

Lele Wang

Chakravarty Reddy Alla Chaitanya

One embodiment of the present invention sets forth a technique for combining a color image and a monochrome image. The technique includes inputting a first set of pixel values from the color image and a second set of pixel values from the monochrome image into a neural network. The technique also includes applying, via a first set of neural network layers, one or more downsampling operations and one or more upsampling operations to feature maps associated with the inputted pixel values to generate a first output image that includes a first set of color values for a first set of pixel locations in the monochrome image. The technique further includes applying a second set of neural network layers to the first set of color values to generate a second output image that includes a second set of color values that is larger than the first set of color values.

Fusion and super-resolution of monochrome and color images using deep neural networks

An image capture system includes an image capture device and an integrated sensor-optical component accessory. The integrated sensor-optical component accessory includes at least one of a microphone, a processor, a motion sensor, or an audio sensor. The image capture device configured to control operation of the integrated sensor-optical component accessory with the image capture device when the integrated sensor-optical component accessory is releasably attached to the image capturing device. The image capture device may include a user interface configurable for operation with the integrated sensor-optical component accessory and the image capture device. The integrated sensor-optical component accessory may draw power from the image capture device. The integrated sensor-optical component accessory may include a power supply. The image capture system may include another integrated sensor-optical component accessory including at least one of a microphone, a processor, a motion sensor, or an audio sensor which is controllable by the image capture device.

Integrated sensor-optical component accessory for image capture device

In one embodiment, a method includes accessing a video captured by cameras which is associated with a first framerate lower than a threshold framerate, for any two adjacent frames of the accessed video: generating a warped frame from the two adjacent frames based on an optical flow associated with the two adjacent frames, determining alignments for the two adjacent frames, respectively, fusing the determined alignments for the two adjacent frames, and generating a reconstructed frame based on the fused alignment, and reconstructing the accessed video based on the any two adjacent frames and their respective reconstructed frames, wherein the reconstructed video is associated with a second framerate higher than the threshold framerate.

Video reconstruction from videos with ultra-low frame-per-second

An image capture device includes a lens barrel having a first end and a second end opposite the first end; one or more inner lenses disposed within the lens barrel; a retaining ring configured to rotatably mount to and be removed from the lens barrel or a body of the image capture device at the first end of the lens barrel; an outer lens configured to be secured by the retaining ring in a position covering the first end of the lens barrel; an O-ring compressible between the outer lens and the lens barrel or the body of the image capture device; and an image sensor mounted within the body of the image capture device at the second end of the lens barrel. The image sensor is configured to capture images based on light incident on the image sensor through the outer lens and the one or more inner lenses.

Lens stack with replaceable outer lens

Systems and methods are disclosed for replaceable outer lenses for use in water. For example, an image capture device may include a lens barrel in a body of the image capture device; a replaceable lens structure mountable on the body of the image capture device, the replaceable lens structure including a first set of two or more stacked lenses, including a first outer lens, and a first retaining ring configured to fasten the first set of two or more stacked lenses against a first end of the lens barrel in a first arrangement, wherein the first set of two or more stacked lenses is configured to collimate light incident on the first outer lens when the first outer lens is underwater at an interface between the lens barrel and the first replaceable lens structure; and an image sensor mounted within the body at a second end of the lens barrel.

Interchangeable lens structures

A digital image capturing device (DICD) that includes: a first integrated sensor-lens assembly (ISLA) facing in a first direction; a second ISLA facing in a second direction generally opposite to the first direction; and a bridge member that is positioned between the first ISLA and the second ISLA, wherein the bridge member is configured as a discrete structure that is separate from the first ISLA and the second ISLA.

Digital image capturing devices including lens modules with integrated sensor-lens assemblies

Systems and methods are disclosed for replaceable outer lenses. For example, an image capture device may include a lens barrel in a body of the image capture device, the lens barrel including multiple inner lenses; a replaceable lens structure that is mountable on the body of the image capture device, the replaceable lens structure including an outer lens and a retaining ring configured to fasten the outer lens in a position covering a first end the lens barrel in a first arrangement and configured to disconnect the outer lens from the body of the image capture device in a second arrangement; and an image sensor mounted within the body at a second end of the lens barrel, the image sensor configured to capture images based on light incident on the image sensor through the outer lens and the multiple inner lenses when the retaining ring is in the first arrangement.

Accessory lens structures for cameras are described. For example, an image capture device may include a mother lens assembly including a first stack of lenses; an image sensor positioned at a first end of the mother lens assembly and configured to detect images based on light incident on the image sensor through the first stack of lenses; a conversion lens assembly including a second stack of lenses, wherein the second stack of lenses is afocal; and a conversion lens mounting apparatus configured to removably attach the conversion lens assembly to the image capture device in a position over a second end of the mother lens assembly, opposite from the image sensor, such that light incident on an outer lens of the second stack of lenses will be refracted through the second stack of lenses and the first stack of lenses to the image sensor.

Accessory lens structures

Video information and sensor information may be obtained. The video information may define spherical video content having a progress length. The spherical video content may define visual content viewable from a point a view as a function of progress through the progress length of the spherical video content. The spherical video content may be captured by one or more image capture devices. The sensor information may characterize capture of the spherical video content. A viewing direction for the spherical video content may be determined based on the sensor information and/or other information. The viewing direction may define a direction of view for the spherical video content from the point of view as the function of progress through the progress length of the spherical video content. The spherical video content may be presented on a display based on the viewing direction.

Systems and methods for identifying viewing directions for video content

A camera mode to use for capturing an image or video is selected by estimating high dynamic range (HDR), motion, and light intensity with respect to a scene of the image or video to capture. An image capture device includes a HDR estimation unit to detect whether HDR is present in a scene of an image to capture, a motion estimation unit to determine whether motion is detected within the scene, and a light intensity estimation unit to determine whether a scene luminance for the scene meets a threshold. A mode selection unit selects a camera mode to use for capturing the image based on output of the HDR estimation unit, the motion estimation unit, and the light intensity estimation unit. An image sensor captures the image according to the selected camera mode.

Automated camera mode selection

Systems and processes for cameras with a reconfigurable lens assembly are described. For example, some methods include automatically detecting that an accessory lens structure has been mounted to an image capture device including a mother lens and an image sensor configured to detect light incident through the mother lens, such that an accessory lens of the accessory lens structure is positioned covering the mother lens; responsive to detecting that the accessory lens structure has been mounted, automatically identifying the accessory lens from among a set of multiple supported accessory lenses; accessing an image captured using the image sensor when the accessory lens structure is positioned covering the mother lens; determining a warp mapping based on identification of the accessory lens; applying the warp mapping to the image to obtain a warped image; and transmitting, storing, or displaying an output image based on the warped image.

Camera with reconfigurable lens assembly

An image capture system for enhanced electronic image stabilization (EIS) includes an image capture device and an adapter lens. The image capture device includes an image sensor, a lens housing, a processor, and a lens assembly that includes a first group of optical elements disposed within the lens housing. The first group of optical elements are used project an image onto the image sensor. The processor performs EIS. The adapter lens is used to enhance EIS of the image capture device. The adapter lens has an adapter lens housing that interfaces with the lens housing. The adapter lens has a second group of optical elements disposed within the adapter lens housing. The second group of optical elements are used to project the image as an image circle on the image sensor.

Wide angle adapter lens for enhanced video stabilization

Systems and methods are disclosed for lens water dispersion. For example, an image capture device may include a lens mounted on a body of the image capture device; an image sensor mounted within the body, behind the lens and configured to detect images based on light incident on the image sensor through the lens; and a dispersion structure around a perimeter of the lens on an external surface of the body, wherein the dispersion structure includes gaps sized to cause capillary action to move water away from the lens, from a first edge of the dispersion structure to a second edge of the dispersion structure.

Growing community of inventors

Hillsborough, CA, United States of America

Ingrid Anda Cotoros