Magic Leap, Inc.

Intel Corporation

Etienne Gregoire Grossmann

John Iselin Woodfill

Gaile Gordon

Zhiheng Jia

Hao Zheng

Nukul Sanjay Shah

Samuel A Miller

Lionel Ernest Edwin

Wenyi Zhao

Michael Robert Johnson

Po-Kang Huang

Brian Christopher Clark

Lei Huang

Robert Dale Tekolste

Ashwin Swaminathan

Jean-Yves Bouguet

Dennis William Strelow

Daniel Roger Dominguez

Marshall Charles Capps

Lei Zhang

Divya Sharma

Kazunori Tanaka

Evan Gregory Levine

Adam Harmat

Yu-Tseh Chi

Eric C Browy

Robert Blake Taylor

Jason A Schaefer

Miller Harry Schuck, Iii

John Monos

Grace Vesom

Nicholas Ihle Morley

Ohad Zohar

Robert Zito

Embodiments provide image display systems and methods for a camera calibration using a two-sided diffractive optical element (DOE). More specifically, embodiments are directed to determining intrinsic parameters of a camera using a single image obtained using a two-sided DOE. The two-sided DOE has a first pattern on a first surface and a second pattern on a second surface. Each of the first and second patterns may be formed by repeating sub-patterns that are lined when tiled on each surface. The patterns on the two-sided DOE are formed such that the brightness of the central intensity peak on the image of the image pattern formed by the DOE is reduced to a predetermined amount.

Camera intrinsic calibration

The systems and methods described can include approaches to calibrate head-mounted displays for improved viewing experiences. Some methods include receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device; receiving data of a first captured image associated with deformed state of the first eyepiece of the head-mounted display device; determining a first transformation that maps the first captured image to the image; and applying the first transformation to a subsequent image for viewing on the first eyepiece of the head-mounted display device.

Compensation for deformation in head mounted display systems

A method for characterizing a digital color camera includes, for each of three primary colors used in a field sequential color virtual image, determining a conversion model for each color using RGB values and the color-measurement values. For each primary color, the method includes illuminating a display device using an input light beam of a primary color having spectral properties representative of a light beam in a virtual image in a wearable device. The method includes capturing, with the digital color camera, an image of the display device, and determining, from the image, RGB values for each primary color. The method includes capturing, with a color-measurement device, a color-measurement value associated with each corresponding primary color at the display device, thereby acquiring a color-measurement value in an absolute color space. A conversion model for each color is determined using RGB values and the color-measurement values.

Method and system for color calibration of an imaging device

Embodiments provide image display systems and methods for one or more camera calibration using a two-sided diffractive optical element (DOE). More specifically, embodiments are directed to determining intrinsic parameters of one or more cameras using a single image obtained using a two-sided DOE. The two-sided DOE has a first pattern on a first surface and a second pattern on a second surface. Each of the first and second patterns may be formed by repeating sub-patterns that are lined when tiled on each surface. The patterns on the two-sided DOE are formed such that the brightness of the central intensity peak on the image of the image pattern formed by the DOE is reduced to a predetermined amount.

Multiple cameras are coupled to a camera rig. Multiple diffractive optical elements (DOEs) are coupled to a DOE prop. Each camera is positioned in an eye box of a DOE and takes an image when the cameras are positioned at a first position, a second position and a third position relative to the DOEs. Three images taken by a each camera at each one of the first position, the second position and third position. The images are transmitted to a processor coupled to the camera rig. For each image, the processor identifies data pairs, each data pair including pixel coordinates of an intensity peak in the image and virtual light source produced by the diffractive optical element that corresponds to the intensity peak, and determines extrinsic parameters of the cameras using the identified data pairs for each image.

System and methods for extrinsic calibration of cameras and diffractive optical elements

A method for calibrating a wearable device includes displaying an image with a plurality of pixels for each of three primary colors using the wearable device, and determining RGB and XYZ values for each of the plurality of pixels. The method includes selecting a subset of the plurality of pixels to form a group of grid points, and dividing the image into a group of tile regions, with each tile region including a grid point. Grid XYZ values are determined for each grid point, based on averaging XYZ values of all pixels in a corresponding tile region, and a grid RGB-to-XYZ conversion matrix is determined for each grid point. The method also includes determining a correction matrix for each grid point by multiplying an inverse of the grid RGB-to-XYZ conversion matrix for the grid with an sRGB-to-XYZ conversion matrix.

Method and system for subgrid calibration of a display device

Techniques are described for calibrating a device having a first sensor and a second sensor. Techniques include capturing sensor data using the first sensor and the second sensor. The device maintains a calibration profile including a translation parameter and a rotation parameter to model a spatial relationship between the first sensor and the second sensor. Techniques include determining a calibration level associated with the calibration profile at a first time. Techniques include determining, based on the calibration level, to perform a calibration process. Techniques include performing the calibration process at the first time by generating one or both of a calibrated translation parameter and a calibrated rotation parameter and replacing one or both of the translation parameter and the rotation parameter with one or both of the calibrated translation parameter and the calibrated rotation parameter.

Calibration of augmented reality device

A method for calibrating a device having a first sensor and a second sensor. The method includes capturing sensor data using the first sensor and the second sensor. The device maintains a calibration profile including a translation parameter and a rotation parameter to model a spatial relationship between the first sensor and the second sensor. The method also includes determining a calibration level associated with the calibration profile at a first time. The method further includes determining, based on the calibration level, to perform a calibration process. The method further includes performing the calibration process at the first time by generating one or both of a calibrated translation parameter and a calibrated rotation parameter and replacing one or both of the translation parameter and the rotation parameter with one or both of the calibrated translation parameter and the calibrated rotation parameter.

Method for calibrating an augmented reality device

Methods, systems, and computer program products for determining, without interrupting a video processing pipeline, whether currently used warping parameters are appropriate for a multi-camera system. New warping parameters may be determined, where these parameters cause the images rectified with these new parameters to have the desired geometric properties. This determination may be made without interrupting the video processing pipeline. Warping parameters may be determined, for example, such that the epipolar lines of the two cameras are horizontal and properly aligned, as defined below, and the metric 3D position of a point observed by both cameras can be determined and is accurate.

Systems, methods, and computer program products for runtime adjustment of image warping parameters in a multi-camera system

Methods, systems, and computer program products to obtain a color image from a color filter array, such as, for example, a color filter array comprising a Bayer pattern. A method of image processing includes, for each pixel location (i,j) and for each color channel in a color image, determining a warped location (i',j&#x2032;) in a color filter array, and determining a color value of the color channel at location (i&#x2032;,j&#x2032;) in the color filter array. The method may further include storing the determined color value. The determining of the color value may include interpolating the color value of the color channel at location (i&#x2032;,j&#x2032;) in the color filter array. In this manner, a single interpolation operation set may be needed and an intermediate image may be avoided, saving memory, reducing processing time, minimizing artifacts, and reducing cost.

Systems, methods, and computer program products for compound image demosaicing and warping

Systems and methods to determine a disparity map using row causal scanline optimization stereo matching are presented. A method includes, for each corresponding pixel P between a pair of input stereo images, and for each considered disparity, determining a basic match cost and a match cost for each of a set of given orientations including an east orientation and one or more other orientations, determining an overall match cost for each pixel at each considered disparity based on a sum of the determined match costs for all considered orientations for each pixel and disparity pair, and determining a resulting disparity for each pixel based on a minimum of the determined overall match costs, where a subset of the determined resulting disparities becomes available prior to completion of the input images being read in, and where the resulting disparities for all pixels are determined in a single pass through the input images.

Systems and methods for row causal scan-order optimization stereo matching

Methods, systems, and computer program products to warp a depth map into alignment with an image, where the image sensor (e.g., camera) responsible for the image and depth sensor responsible for an original depth map are separated in space. In an embodiment, the warping of the depth map may be started before the original depth map has been completely read. Moreover, data from the warped depth map may be made available to an application before the entire warped depth map has been completely generated. Such a method and system may improve the speed of the overall process and/or reduce memory requirements.

Growing community of inventors

Menlo Park, CA, United States of America

Etienne Gregoire Grossmann