Qualcomm Incorporated

Bijan Forutanpour

Ning Bi

Sebastien Mounier

Jonathan Kies

Sairam Sundaresan

Phi Hung Nguyen

Wesley James Holland

Ramesh Chandrasekhar

Daniel James Guest

Phi Hung Le Nguyen

Shankar Thagadur Shivappa

Michel Adib Sarkis

Ye-Kui Wang

Martin Renschler

Tao Zhang

Feng Guo

Cullum Baldwin

Yingyong Qi

Jianfeng Ren

Gautham Datta Mayyuri

Michael J Mager

Michael Mager

Gautham Mayyuri

An imaging system can receive an image of a portion of an environment. The environment can include an object, such as a hand or a display. The imaging device can identify a data stream from an external device, for instance by detecting the data stream in the image or by receiving the data stream wirelessly from the external device. The imaging device can detect a condition based on the image and/or the data stream, for instance by detecting that the object is missing from the image, by detecting that a low resource at the imaging device, and/or by detecting visual media content displayed by a display in the image. Upon detecting the condition, imaging device automatically determines a location of the object (or a portion thereof) using the data stream and/or the image. The imaging device generates and/or outputs content that is based on the location of the object.

Collaborative tracking

Examples are described of marking specified regions of stored image frame buffer data in an image frame buffer. An imaging system can read the specified regions of the image frame buffer to identify whether the marking has been overwritten or not. The imaging system can thus efficiently identify how much of the image frame buffer has been overwritten with data from a new image frame. Based on this, the imaging system can retrieve partial image frame data from the image frame buffer and can process the partial image frame data, for instance to composite the partial image frame data with virtual content and/or to perform distortion compensation. The processed partial image frame data can be uploaded to a display buffer and displayed by a display, either as-is or once more of the frame is captured and processed. The imaging system can also perform auto-exposure using the partial image frame data.

Low latency frame delivery

Systems and techniques are described for extended reality (XR) operations. An XR system displays virtual content using a display according to display settings. The display settings can identify, for instance, a position, orientation, and/or size of the virtual content as displayed. The environment can be viewable using the display as the virtual content is displayed by the display, for example using a see-through display or a pass-through display. The imaging system can determine, based on one or more attributes of one or both eyes of the user of the imaging system, an extent of perception of the virtual content that is displayed using the display by the user. The attributes can identify, for instance, eye position, eye movement, pupil dilation, saccades, fixations, blinking, and/or squinting. The XR system can determine, based on the extent of perception of the virtual content by the user, a modification to the display settings.

Dynamic content presentation for extended reality systems

The techniques disclosed herein include a first device including one or more processors configured to detect a selection of at least one target object external to the first device, and initiate a channel of communication between the first device and a second device associated with the at least one target object external to the first device. The one or more processors may be configured to receive audio packets, from the second device, in response to the selection of at least one target object external to the device, decode the audio packets, received from the second device, to generate an audio signal. The one or more processors may be configured to output the audio signal based on the selection of the at least one target object external to the first device. The first device includes a memory, coupled to the one or more processors, configured to store the audio packets.

Rendering of sounds associated with selected target objects external to a device

Systems, methods, and non-transitory media are provided for generating obfuscated control interfaces for extended reality (XR) experiences. An example method can include determining a pose of an XR device within a mapped scene of a physical environment associated with the XR device; rendering a virtual control interface within the mapped scene according to a configuration including a first size, a first position relative to the pose of the XR device, a first ordering of input elements, and/or a first number of input elements; and adjusting the configuration of the virtual control interface based on a privacy characteristic of data associated with the virtual control interface and/or characteristics of the physical environment associated with the XR device, the adjusted configuration including a second size, a second ordering of input elements, a second number of input elements, and/or a second position relative to the pose of the XR device and/or first position.

Obfuscated control interfaces for extended reality

The present disclosure relates to methods and apparatus for display or graphics processing. Aspects of the present disclosure can calculate first location information based on a first image mesh. Further, aspects of the present disclosure can determine a first UV image based on the calculated first location information. Additionally, aspects of the present disclosure can calculate second location information based on a second image mesh and the determined first UV image. Aspects of the present disclosure can also determine a second UV image based on the calculated second location information. Aspects of the present disclosure can also generate a third image mesh based on the determined second UV image. Also, aspects of the present disclosure can calculate third location information based on the third image mesh and a camera image. Aspects of the present disclosure can also determine a display image based on the calculated third location information.

Methods and apparatus for multiple lens distortion correction

Systems, methods, and non-transitory media are provided for generating private control interfaces for extended reality (XR) experiences. An example method can include determining a pose of an XR device within a mapped scene of a physical environment associated with the XR device; detecting a private region in the physical environment and a location of the private region relative to the pose of the XR device, the private region including an area estimated to be within a field of view (FOV) of a user of the XR device and out of a FOV of a person in the physical environment, a recording device in the physical environment, and/or an object in the physical environment; based on the pose of the XR device and the location of the private region, mapping a virtual private control interface to the private region; and rendering the virtual private control interface within the private region.

Private control interfaces for extended reality

The techniques disclosed herein include a first device for reading one or more tags in metadata, the first device including one or more processors configured to receive metadata, from a second device, wirelessly connected via a sidelink channel to the first device. The one or more processors may also be configured to read the metadata, received from the second device to extract one or more tags representative of audio content, and identify audio content based on the one or more tags, and output the audio content. The first device may also include a memory, coupled to the one or more processors, configured to store the metadata.

Media exchange between devices

The techniques disclosed herein include a first device for receiving a communication signal from a second device, the first device including one or more processors configured to receive, in the communication signal, packets that represent a virtual image as part of a virtual teleportation of one or more visual objects embedded in the virtual image. The one or more processors may be configured to decode the packets that represent the virtual image, and output the virtual image at a physical location within a fixed environment. The first device may also include a memory configured to store the packets that represent the virtual image as part of the virtual teleportation of one or more visual objects embedded in the virtual image.

Virtual teleportation in fixed environments

An example method includes processing a file including fisheye video data, the file including a syntax structure including a plurality of syntax elements that specify attributes of the fisheye video data, wherein the plurality of syntax elements includes: a first syntax element that explicitly indicates whether the fisheye video data is monoscopic or stereoscopic, and one or more syntax elements that implicitly indicate whether the fisheye video data is monoscopic or stereoscopic; determining, based on the first syntax element, whether the fisheye video data is monoscopic or stereoscopic; and rendering, based on the determination, the fisheye video data as monoscopic or stereoscopic.

High-level signaling for fisheye video data

Provided are systems, methods, and computer-readable medium for including parameters that describe fisheye images in a 360-degree video with the 360-degree video. The 360-degree video can then be stored and/or transmitted as captured by the omnidirectional camera, without transforming the fisheye images into some other format. The parameters can later be used to map the fisheye images to an intermediate format, such as an equirectangular format. The intermediate format can be used to store, transmit, and/or display the 360-degree video. The parameters can alternatively or additionally be used to map the fisheye images directly to a format that can be displayed in a 360-degree video presentation, such as a spherical format.

Growing community of inventors

San Diego, CA, United States of America

Bijan Forutanpour