Apple Inc.

Marc-Andre Chenier

Bertrand Nepveu

Yves Millette

Yan Cote

Tomlinson M Holman

Anthony Ghannoum

Marc-Olivier Lepage

Vincent Chapdelaine-Couture

Simon Fortin-Deschênes

Sandy J Carter

Simon Fortin-Deschenes

Simon Fortin-deschênes

Irving Lustigman

In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.

Method and device for latency reduction of an image processing pipeline

The present disclosure relates to techniques for presenting a combined view of a virtual environment and a real environment in response to detecting a transition event associated with an object in the real environment. While presenting the combined view, if an input of a first type is detected, the combined view is adjusted by increasing the visibility of imagery of the virtual environment and decreasing the visibility of imagery of the real environment. If an input of a second type is detected, the combined view is adjusted by decreasing the visibility of the imagery of the virtual environment and increasing the visibility of the imagery of the real environment.

Inserting imagery from a real environment into a virtual environment

The present disclosure relates to techniques for transitioning between imagery and sounds of two different environments, such as a virtual environment and a real environment. A view of a first environment and audio associated with the first environment are provided. In response to detecting a transition event, a view of the first environment combined with a second environment is provided. The combined view includes imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value. In addition, in response to detecting a transition event, the first environment audio is mixed with audio associated with the second environment.

Transitioning between imagery and sounds of a virtual environment and a real environment

The present disclosure relates to techniques for inserting imagery from a real environment into a virtual environment. While presenting (e.g., displaying) the virtual environment at an electronic device, a proximity of the electronic device to a physical object located in a real environment is detected. In response to detecting that the proximity of the electronic device to the physical object is less than a first threshold distance, imagery of the physical object is isolated from other imagery of the real environment. The isolated imagery of the physical object is inserted into the virtual environment at a location corresponding to the location of the physical object in the real environment. The imagery of the physical object has a first visibility value associated with the proximity of the electronic device to the physical object.

A Head-Mounted Display with camera sensors to perform chroma keying in a mixed reality context is presented. Low latency is achieved by embedding the processing in the HMD itself, specifically, format camera images, detect the selected color range and make a composite with the virtual content.

Low latency chroma keying embedded in a head-mounted display for mixed reality

A method includes obtaining a first plurality of image frames that are provided by an image sensor at a first output rate. The method includes determining a GPU temporal processing value characterizing the GPU processing of the first plurality of image frames. The GPU temporal processing value is a function of an amount of information associated with the first plurality of image frames. The method includes determining an operation value associated with the image sensor based on a function of the GPU temporal processing value. The operation value modifies the operation of the image sensor. The method includes changing, using a controller, the operation of the image sensor as a function of the operation value so that the image sensor provides a second plurality of image frames at a second output rate.

Controlling an image sensor based on GPU temporal processing values

In some implementations, a method of reducing latency associated with an image read-out operation is performed at a device including one or more processors, non-transitory memory, an image processing architecture, and an image capture device. The method includes: obtaining first image data corresponding to a physical environment; reading a first slice of the first image data into an input buffer; performing processing operations on the first slice of the first image data to obtain a first portion of second image data; reading a second slice of the first image data into the input buffer; performing the image processing operations on the second slice of the first image data to obtain a second portion of the second image data; and generating an image frame of the physical environment based at least in part on the first and second portions of the second image data.

Growing community of inventors

Saint-Hyacinthe, Canada

Marc-Andre Chenier