Apple Inc.

Vrvana Inc.

Bertrand Nepveu

Marc-Andre Chenier

Yan Cote

Yves Millette

Tomlinson M Holman

Anthony Ghannoum

Vincent Chapdelaine-Couture

Marc-Olivier Lepage

Jonathan P Ive

Alan C Dye

Stephen Olivier Lemay

Julian Jaede

Fletcher R Rothkopf

Cheng Kai Chen

Jeremy C Franklin

William A Sorrentino, Iii

Jason C Sauers

Geoffrey Grant Stahl

Wey-Jiun Lin

Arthur Yasheng Zhang

Ivan S Maric

Phil M Hobson

Graham Butler Myhre

Michael J Rockwell

Forrest C Wang

Yoonhoo Jo

Muhammad F Hossain

Jia Tao

Marinus Meursing

Yanghai Tsin

Kathrin Berkner Cieslicki

Tom Sengelaub

Jan K Quijalvo

Simon Fortin-Deschênes

Julian Hoenig

Jason L Slupeiks

Simon Fortin-Deschenes

Emmanuel Piuze-Phaneuf

Sandy J Carter

Peter Kaufmann

Irving Lustigman

Simon Fortin-deschênes

Thibaut Weise

Branko Petljanski

James W Vandyke

Hesam Najafi

A head-mountable display device includes a housing defining a front opening and a rear opening, a display screen disposed in the front opening, a display assembly disposed in the rear opening, a first securement strap coupled to the housing, the first securement strap including a first electronic component, a second securement strap coupled to the housing, the second securement strap including a second electronic component, and a securement band extending between and coupled to the first securement strap and the second securement strap.

Head mountable display

The method includes: obtaining a first image of an environment from a first image sensor associated with first intrinsic parameters; performing a warping operation on the first image according to perspective offset values to generate a warped first image in order to account for perspective differences between the first image sensor and a user of the electronic device; determining an occlusion mask based on the warped first image that includes a plurality of holes; obtaining a second image of the environment from a second image sensor associated with second intrinsic parameters; normalizing the second image based on a difference between the first and second intrinsic parameters to produce a normalized second image; and filling a first set of one or more holes of the occlusion mask based on the normalized second image to produce a modified first image.

Method and device for multi-camera hole filling

In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.

Method and device for latency reduction of an image processing pipeline

The present disclosure relates to techniques for presenting a combined view of a virtual environment and a real environment in response to detecting a transition event associated with an object in the real environment. While presenting the combined view, if an input of a first type is detected, the combined view is adjusted by increasing the visibility of imagery of the virtual environment and decreasing the visibility of imagery of the real environment. If an input of a second type is detected, the combined view is adjusted by decreasing the visibility of the imagery of the virtual environment and increasing the visibility of the imagery of the real environment.

Inserting imagery from a real environment into a virtual environment

The present disclosure relates to techniques for inserting imagery from a real environment into a virtual environment. While presenting (e.g., displaying) the virtual environment at an electronic device, a proximity of the electronic device to a physical object located in a real environment is detected. In response to detecting that the proximity of the electronic device to the physical object is less than a first threshold distance, imagery of the physical object is isolated from other imagery of the real environment. The isolated imagery of the physical object is inserted into the virtual environment at a location corresponding to the location of the physical object in the real environment. The imagery of the physical object has a first visibility value associated with the proximity of the electronic device to the physical object.

A Head-Mounted Display with camera sensors to perform chroma keying in a mixed reality context is presented. Low latency is achieved by embedding the processing in the HMD itself, specifically, format camera images, detect the selected color range and make a composite with the virtual content.

Low latency chroma keying embedded in a head-mounted display for mixed reality

A method includes obtaining a first plurality of image frames that are provided by an image sensor at a first output rate. The method includes determining a GPU temporal processing value characterizing the GPU processing of the first plurality of image frames. The GPU temporal processing value is a function of an amount of information associated with the first plurality of image frames. The method includes determining an operation value associated with the image sensor based on a function of the GPU temporal processing value. The operation value modifies the operation of the image sensor. The method includes changing, using a controller, the operation of the image sensor as a function of the operation value so that the image sensor provides a second plurality of image frames at a second output rate.

Controlling an image sensor based on GPU temporal processing values

In some implementations, a method of reducing latency associated with an image read-out operation is performed at a device including one or more processors, non-transitory memory, an image processing architecture, and an image capture device. The method includes: obtaining first image data corresponding to a physical environment; reading a first slice of the first image data into an input buffer; performing processing operations on the first slice of the first image data to obtain a first portion of second image data; reading a second slice of the first image data into the input buffer; performing the image processing operations on the second slice of the first image data to obtain a second portion of the second image data; and generating an image frame of the physical environment based at least in part on the first and second portions of the second image data.

Generating a representation of a scene includes detecting an indication to capture sensor data to generate a virtual representation of a scene in a physical environment at a first time, in response to the indication obtaining first sensor data from a first capture device at the first time, obtaining second sensor data from a second capture device at the first time, and combining the first sensor data and the second sensor data to generate the virtual representation of the scene.

Automatically capturing a moment

A headset system connectable to a video source for receiving a video signal through a wired or wireless communication link comprises a head mounted device, a user interface and a base station. The head mounted device displays a video image to the user via an integrated video display and also comprises an integrated data image processor and optical lenses positioned in front the video display. The data image processor effects high speed real time modification of an image frame of the video signal. The video display is in communication with the data image processor for displaying processed images from the digital image processor. The user interface allows for building a user profile and storing the user profile in the memory of the digital image processor. The base station comprises an input interface connectable to the communication link, and an output interface for forwarding the communication signal to the video display.

Method, digital image processor and video display system for digitally processing a video signal

Growing community of inventors

Montreal, Canada

Bertrand Nepveu