Intel Corporation

Apple Inc.

Intel Corportation

Gowri Somanath

Oscar Nestares

Daniel R Kurz

Tobias Holl

Yong Jiang

Yi Wu

Jiajie Yao

Christos G Bampis

Kalpana Seshadrinathan

Basel Salahieh

Ginni Grover

Karthik Natarajan

A mechanism is described for facilitating cinematic space-time view synthesis in computing environments according to one embodiment. A method of embodiments, as described herein, includes capturing, by one or more cameras, multiple images at multiple positions or multiple points in times, where the multiple images represent multiple views of an object or a scene, where the one or more cameras are coupled to one or more processors of a computing device. The method further includes synthesizing, by a neural network, the multiple images into a single image including a middle image of the multiple images and representing an intermediary view of the multiple views.

Cinematic space-time view synthesis for enhanced viewing experiences in computing environments

Various implementations disclosed herein include devices, systems, and methods that use light probes to facilitate the display of virtual objects in 3D environments. A light probe provides lighting information that describes light incident on a point in space in a 3D environment. For example, a light probe may describe such incident light using an environment map. Such lighting information can be used to provide realistic appearances for objects placed at or near light probe locations in the 3D environment. Implementations disclosed herein determine the light probe locations in real-time or other 3D environments that are generated based on a live physical environment. A digital representation of the live physical environment is used to determine where to position the light probes, how many light probes to use, and/or various light probe attributes.

Light probe placement for displaying objects in 3D environments on electronic devices

Various implementations disclosed herein include devices, systems, and methods that render a reflective surface of a computer-generated reality ('CGR') object based on synthesis in a CGR environment. In order to render a reflective surface of the CGR object, one exemplary implementation involves synthesizing an environment map of a CGR environment representing a portion of a physical scene based on observed characteristics of the physical scene. In an implementation, generation of a complete environment map includes identifying pixels of the environment map with no corresponding texture and generating synthesized texture based on textural information associated with one or more camera images of the physical scene. In an implementation, a CGR object is rendered in the CGR environment, wherein an appearance of a reflective surface of the CGR object is determined based on the complete environment map of the CGR environment.

Environment synthesis for lighting an object

Various implementations disclosed herein include devices, systems, and methods that complete content for a missing part of an image of an environment. For example, an example process may include obtaining an image including defined content and missing parts for which content is undefined, determining a spatial image transformation for the image based on the defined content and the missing parts of the image, altering the image by applying the spatial image transformation, and completing the altered image.

Partial image completion

Implementations disclosed herein provide systems and methods that use classification-based machine learning to generate perceptually-plausible content for a missing part (e.g., some or all) of an image. The machine learning model may be trained to generate content for the missing part that appears plausible by learning to generate content that cannot be distinguished from real image content, for example, using adversarial loss-based training. To generate the content, a probabilistic classifier may be used to select color attribute values (e.g., RGB values) for each pixel of the missing part of the image. To do so, a pixel color attribute is segmented into a number of bins (e.g., value ranges) that are used as classes. The classifier determines probabilities for each of the bins of a color attribute for each pixel and generates the content by selecting the bin having the highest probability for each color attribute for each pixel.

Classification for image creation

An apparatus, method, and machine-readable medium for health monitoring and response are described herein. The apparatus includes a processor and a number of sensors configured to collect data corresponding to a user of the device. The apparatus also includes a health monitoring and response application, at least partially including hardware logic. The hardware logic of the health monitoring and response application is to test the data collected by any of the sensors to match the collected data with a predetermined health condition, determine a current health condition of the user based on the predetermined health condition that matches the collected data, and automatically perform an action based on the current health condition of the user.

Device for health monitoring and response

Techniques are provided for segmentation of objects in video frames. A methodology implementing the techniques according to an embodiment includes receiving image frames, including an initial reference frame, and receiving a mask to outline a region in the reference frame that contains the object to be segmented. The method also includes calculating Gaussian mixture models associated with both the masked region and a background region external to the masked region. The method further includes segmenting the object from a current frame based on a modelling of the pixels within an active area of the current frame as a Markov Random Field of nodes for cost minimization. The costs are based in part on the Gaussian mixture models. The active area is based on the segmentation of a previous frame and on an estimation of optical flow between the previous frame and the current frame.

Segmentation of objects in videos using color and depth information

Image scene labeling with 3D image data. A plurality of pixels of an image frame may be label based at least on a function of pixel color and a pixel depth over the spatial positions within the image frame. A graph-cut technique may be utilized to optimize a data cost and neighborhood cost in which at least the data cost function includes a component that is a dependent on a depth associated with a given pixel in the frame. In some embodiments, in the MRF formulation pixels are adaptively merged into pixel groups based on the constructed data cost(s) and neighborhood cost(s). These pixel groups are then made nodes in the directed graphs. In some embodiments, a hierarchical expansion is performed, with the hierarchy set up within the label space.

Fast MRF energy optimization for solving scene labeling problems

Techniques are provided for perception enhancement of light fields (LFs) for use in integral display applications. A methodology implementing the techniques according to an embodiment includes receiving one or more LF views and a disparity map associated with each LF view. The method also includes quantizing the disparity map into planes, where each plane is associated with a selected range of depth values. The method further includes slicing the LF view into layers, where each layer comprises pixels of the LF view associated with one of the planes. The method further includes shifting each of the layers in a lateral direction by an offset distance. The offset distance is based on a viewing angle associated with the LF view and further based on the depth values of the associated plane. The method also includes merging the shifted layers to generate a synthesized LF view with increased parallax.

Light field perception enhancement for integral display applications

A system, article, and method of sub-pixel accuracy 3D measurement using multiple images.

Method and system of sub pixel accuracy 3D measurement using multiple images

Techniques are provided for fusion of image frames to generate panoramic background images using color and depth data provided from a 3D camera. An example system may include a partitioning circuit configured to partition an image frame into segments and objects, the segments comprising a group of pixels sharing common features associated with the color and depth data, the objects comprising one or more related segments. The system may also include an object consistency circuit configured to assign either 2D or 3D transformation types to each of the segments and objects to transform them to a co-ordinate system of a reference image frame. The system may further include a segment recombination circuit to combine the transformed objects and segments into a transformed image frame and an integration circuit to integrate the transformed image frame with the reference image frame to generate the panoramic image.

Fusion of panoramic background images using color and depth data

A system, article, and method of lens shift correction for a camera array.

Growing community of inventors

Santa Clara, CA, United States of America

Gowri Somanath