Marvell International Limited

Qualcomm Incorporated

Apple Inc.

Intel Corporation

Moinul H Khan

Anitha Kona

Jim Chen Chou

Mark Norman Fullerton

Bradley C Aldrich

Aleksandr M Movshovich

Arthur Yasheng Zhang

Tobias Eble

Bohuslav Rychlik

Hao Pan

Martin Renschler

Subbarao Palacharla

Sorin Constantin Cismas

Nhon Toai Quach

Yanru Li

Nicolas P Bonnier

Feng Wang

Serag Gadelrab

Alain Artieri

Chaohao Wang

Milivoje M Aleksic

Paolo Sacchetto

Raffi A Bedikian

Sheng Zhang

Anselm Grundhoefer

Mohamed Selim Ben Himane

Nigel C Paver

Chinchuan Andrew Chiu

Andrew Edmund Turner

George Patsilaras

Meghal Varia

Paul V Johnson

Sreeraman Anantharaman

Yunhui Hou

Ning Bi

Azzedine Touzni

Alexandre Da Veiga

Chia-Yuan Teng

Bruno M Sommer

Mark Michael Schaffer

Christopher Kong Yee Chun

Jaya Prakash Subramaniam Ganasan

Ronald Alton

Colin Christopher Sharp

Idreas Mir

Gheorghe Calin Cascaval

Mehrad Tavakoli

Christopher T Foulds

Timothy R Cahalan

Vinod Chamarty

Priya N Vaidya

John P Brizek

Jian Zhou

David Wheeler

Simon Booth

Siddharth Shankar Hazra

Mriganka Mondal

Ye Cong

Hariprasad Puthukkootil Rajagopal

Haw-Jing Lo

Michael Drop

Maurice Franklin Ribble

Holly Gerhard

Kyle Ernewein

Raghu Sankuratri

Ralf Habel

Pedro Manuel Da Silva Quelhas

Tom Longo

Cody J White

Kedar Bhole

Kayla L Chalmers

Srikanth Rengarajan

Marwan Hussein

Randall R Rauwendaal

Kris Tiri

Steven S Thomson

Anwar Q Rohillah

James H Zhou

Ramesh Chandrasekhar

Venu Madhav Duggineni

Joshua Stubbs

Rashmi Kulkarni

Yohan Rajan

Ali Iranli

Sudeep Ravi Kottilingal

Jia Bao

Raghavendra Nagaraj

Jeffrey S Norris

Bennett S Wilburn

Assaf Menachem

Nan Zhu

King-Chung Lai

Virat Deepak

William W Sprague

Tao Xue

Cristian Duroiu

Vinodh Ramesh Cuppu

Ricky Wai Kit Yuen

Robert Nicholson Gibson

Simon Fortin-Deschenes

An electronic device may have a camera and a display. The display may be configured to display virtual reality content for a user in which no real-world content from the camera is displayed or mixed reality content in which a combination of real-world content from the camera and overlaid virtual reality content is displayed. Control circuitry in the device may adjust the display and camera while transitioning between virtual reality and mixed reality modes. The control circuitry may reconfigure the camera to exhibit a desired frame rate immediately upon transitioning from virtual reality mode to mixed reality mode. Transitions between modes may be accompanied by smooth transitions between frame rates to avoid visible artifacts on the display. The camera frame rate may be synchronized to the display frame rate for at least part of the transition between the virtual reality and mixed reality modes.

Electronic device with coordinated camera and display operation

An electronic device includes a wireless transceiver configured to receive content primitives via a wireless communication channel. The electronic device also includes control circuitry control circuitry coupled to the wireless transceiver, and configured to perform content provisioning operations based on the received content primitives, wherein the content provisioning operations comprise generating content image data and transmitting the content image data to the wireless communication channel using the wireless transceiver. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that decrease an amount of content image data conveyed by the wireless transceiver to the wireless communication channel.

Electronic device content provisioning adjustments based on wireless communication channel bandwidth condition

In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

Dynamic foveated pipeline

A method includes obtaining rendered image data that includes a representation of an object for display using a see-through display. The see-through display permits ambient light from a physical environment through the see-through display. The method includes sensing a plurality of light superposition characteristic values associated with the ambient light that quantifies the ambient light. The method includes determining a plurality of display correction values associated with the electronic device based on the plurality of light superposition characteristic values and predetermined display characteristics of the representation of the object. The method includes generating, from the rendered image data, display data for the see-through display in accordance with the plurality of display correction values in order to satisfy the predetermined display characteristics of the representation of the object within a performance threshold.

Modifying rendered image data based on ambient light from a physical environment

A method includes obtaining first pass-through image data characterized by a first pose. The method includes obtaining respective pixel characterization vectors for pixels in the first pass-through image data. The method includes identifying a feature of an object within the first pass-through image data in accordance with a determination that pixel characterization vectors for the feature satisfy a feature confidence threshold. The method includes displaying the first pass-through image data and an AR display marker that corresponds to the feature. The method includes obtaining second pass-through image data characterized by a second pose. The method includes transforming the AR display marker to a position associated with the second pose in order to track the feature. The method includes displaying the second pass-through image data and maintaining display of the AR display marker that corresponds to the feature of the object based on the transformation.

Methods and devices for detecting and identifying features in an AR/VR scene

A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

Mobile device content provisioning adjustments based on wireless communication channel bandwidth condition

A method includes obtaining an image via an image sensor, and identifying, within the image, a physical object represented by a portion of the image. The method includes determining, based on the image, a visual feature characterizing the physical object. The method includes warping, based on the visual feature satisfying a first feature criterion, the portion of the image according to a first warping function that is based on the first feature criterion and a distance between the electronic device and a reference point. The method includes warping, based on the visual feature satisfying a second feature criterion that is different from the first feature criterion, the portion of the image according to a second warping function that is based on the second feature criterion and the distance between the electronic device and the reference point.

Adaptive image warping based on object and distance information

Various implementations disclosed herein include devices, systems, and methods that dynamically-size zones used in foveated rendering of content that includes text. In some implementations, this involves adjusting the size of a first zone, e.g., a foveated gaze zone (FGZ), based on the apparent size of text from a viewpoint. For example, a FGZ may be increased or decreased in width, height, diameter, or other size attribute based on determining an angle subtended by one or more individual glyphs of the text from the viewpoint. Various implementations disclosed herein include devices, systems, and methods that select a text-rendering algorithm based on a relationship between (a) the rendering resolution of a portion of an image corresponding to a part of a glyph and (b) the size that the part of the glyph will occupy in the image.

Rendering computer-generated reality text

Some implementations provide improved user experiences on head mounted devices (HMDs) that provide near eye viewing, e.g., HMDs that display distorted images and provide lenses that undistort the images for the user. The images are produced using distortion that is corrected dynamically based on context to conserve device resources. To do so, a context associated with a state of the user, the HMD, or content being viewed on the HMD is tracked during the user experience. For example, the device may predict pupil position, eye state, eye gaze direction, or eye fixation, content type, connection mode, and other context. The device uses the tracked context to determine how to correct distortion for the images at different points during the user experience. For example, new distortion corrections may be computed and used while the user's gaze is moving and previously-determined distortion corrections may be used while the user's gaze is fixed.

Context aware dynamic distortion correction

In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

Error concealment for a head-mountable device

A method includes obtaining an image via an image sensor. The method includes determining a first perceptual quality value that is associated with a first portion of the image. The method includes determining a first image perceptual quality warping function that is based on the first perceptual quality value and an image warping map. The first image perceptual quality warping function is characterized by a first warping granularity level that is a function of the first perceptual quality value. The method includes warping the first portion of the image according to the first image perceptual quality warping function.

Variable-granularity based image warping

One exemplary implementation involves performing operations at an electronic device with one or more processors and a computer-readable storage medium. The device establishes a wireless communication link with a host device. The device receives, from the host device, a left eye frame and a right eye frame via a sequence of left eye frame transmissions and right eye frame transmissions. The device switches data transmissions schemes according to wireless commination link quality or eye gaze tracking. Adjusting transmission format based on transmission quality of the wireless communication link allows the devices to take advantage of greater bandwidth when available to save power. An additional transmission format is based on alternately transmitting left eye and right eye frames for very low bandwidth.

Adaptive wireless transmission schemes

Growing community of inventors

San Jose, CA, United States of America

Moinul H Khan