Qualcomm Incorporated

Gerhard Reitmayr

Dieter Schmalstieg

Serafin Diaz Spindola

Alessandro Mulloni

Mehrad Tavakoli

Daniel Wagner

Ajit Deepak Gupte

Robert S Tartz

Kiran Kumar Somasundaram

Scott David Beith

Pushkar Gorur Sheshagiri

Abhijeet Bisain

Bardia Fallah Behabadi

Tobias Langlotz

Ning Bi

Christopher G Lott

Bojan Vrcelj

Michel Adib Sarkis

Wesley James Holland

Qi Pan

Ke-Li Cheng

Simon Peter William Booth

Chiranjib Choudhuri

Kuang-Man Huang

Clemens Arth

Aliakbar Aghamohammadi

Casimir Matthew Wierzynski

Huang Huang

Injoon Hong

Hartmut Seichter

Sarah Paige Gibson

Christian Pirchheim

Saurav Agarwal

Ashar Ali

Shayegan Omidshafiei

Thanh Quoc Nguyen

Peerapol Tinnakornsrisuphap

Prashanth Haridas Hande

Chih-Ping Li

Ananthapadmanabhan Arasanipalai Kandhadai

Alok Govil

Victor Hokkiu Chan

Onkar Jayant Dabeer

Ajit Venkat Rao

Gokce Dane

Mahesh Ramachandran

Peter Hans Rauber

Ravi Teja Sukhavasi

Radhika Dilip Gowaikar

Michael Gervautz

Erick Mendez Mendez

Upal Mahbub

Zsolt Szalavari

Hussein Metwaly Saad

Prince Gupta

Tom Edward Botterill

Youngmin Park

Raphael Grasset

Daniel Knoblauch

Emilio Maggio

Young Hoon Lee

Chayan Sharma

Joseph Daniel Garvey

Samuel Sunarjo

Srujan Babu Nandipati

Gokee Dane

Vladimir Slobodyanyuk

Phillip Grasmug

Raphael David Andre Grasset

Winston Yll

Mathaus Zingerle

Gerhard Schall

Systems and techniques are described herein for processing images. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a first processor receives an image of an environment captured by an image sensor, identifies features depicted in the image, and generates descriptors for the features. The first processor sends the descriptors to a second processor, which may be more powerful than the first processor. The second processor receives the descriptors. The second processor associates the plurality of features with a map of the environment based on at least a subset of the plurality of descriptors. For example, the second processor can track at least a subset of the features based on at least a subset of the descriptors and based on feature information from one or more additional images of the environment.

Feature processing in extended reality systems

Systems and techniques are provided for determining and applying corrected poses in digital content experiences. An example method can include receiving, from one or more sensors associated with an apparatus, inertial measurements and one or more frames of a scene; based on the one or more frames and the inertial measurements, determining, via a first filter, an angular and linear motion of the apparatus and a gravity vector indicating a direction of gravitational force interacting with the apparatus; when a motion of the apparatus is below a threshold, determining, via a second filter, an updated gravity vector indicating a direction of gravitational force interacting with the apparatus; determining, based on the updated gravity vector, parameters for aligning an axis of the scene with a gravity direction in a real-world spatial frame; and aligning, using the parameters, the axis of the scene with the gravity direction in the real-world spatial frame.

Pose correction for digital content

Systems and techniques are provided for modeling three-dimensional (3D) meshes using images. An example method can include receiving, via a neural network system, an image of a target and metadata associated with the image and/or a device that captured the image; determining, based on the image and metadata, first 3D mesh parameters of a first 3D mesh of the target, the first 3D mesh parameters and first 3D mesh corresponding to a first reference frame associated with the image and/or the device; and determining, based on the first 3D mesh parameters, second 3D mesh parameters for a second 3D mesh of the target, the second 3D mesh parameters and second 3D mesh corresponding to a second reference frame, the second reference frame including a 3D coordinate system of a real-world scene where the target is located.

Modeling objects from monocular camera outputs

Techniques are provided for generating three-dimensional models of objects from one or more images or frames. For example, at least one frame of an object in a scene can be obtained. A portion of the object is positioned on a plane in the at least one frame. The plane can be detected in the at least one frame and, based on the detected plane, the object can be segmented from the plane in the at least one frame. A three-dimensional (3D) model of the object can be generated based on segmenting the object from the plane. A refined mesh can be generated for a portion of the 3D model corresponding to the portion of the object positioned on the plane.

Object scanning using planar segmentation

Examples are described for processing images to mask dynamic objects out of images to improve feature tracking between images. A device receives an image of an environment captured by an image sensor. The image depicts at least a static portion of the environment and a dynamic object in the environment. The device identifies a portion of the image that includes a depiction of the dynamic object. For example, the device can detect a bounding box around the dynamic object, or can detect which pixels in the image correspond to the dynamic object. The device generates a masked image at least by masking the portion of the image. The device identifies features in the masked image, and uses the features from the masked image for feature tracking from other images of the environment, masked or otherwise. The device can use this feature tracking for mapping, localization, and/or relocation.

Object segmentation and feature tracking

Systems, methods, and computer-readable media are provided for providing pose estimation in extended reality systems. An example method can include tracking, in a lower-power processing mode using a set of lower-power circuit elements on an integrated circuit, a position and orientation of a computing device during a lower-power processing period, the set of lower-power circuit elements including a static random-access memory (SRAM); suspending, based on a triggering event, the tracking in the lower-power processing mode; initiating a higher-power processing mode for tracking the position and orientation of the computing device during a higher-power processing period; and tracking, in the higher-power processing mode using a set of higher-power circuit elements on the integrated circuit and a dynamic random-access memory (DRAM), the position and orientation of the computing device during the higher-power processing period.

Pose estimation in extended reality systems

This disclosure provides systems, methods, and devices for wireless communication that support enhanced beam management using extended reality (XR) perception data. In a first aspect, a method of wireless communication includes establishing a communication connection between a user equipment (UE) and a serving base station using a current serving beam selected by the UE from a plurality of available beams paired with a serving base station beam. The method further includes obtaining, perception information from one or more extended reality sensors associated with the UE and determining, in response to detection of UE movement, a transpositional representation of the movement using the perception information. The UE may then select a new serving beam in accordance with the transpositional representation. Other aspects and features are also claimed and described.

Exploiting extended reality perception data for beam management

Systems and techniques are provided for modeling three-dimensional (3D) meshes using multi-view image data. An example method can include determining, based on a first image of a target, first 3D mesh parameters for the target corresponding to a first coordinate frame; determining, based on a second image of the target, second 3D mesh parameters for the target corresponding to a second coordinate frame; determining third 3D mesh parameters for the target in a third coordinate frame, the third 3D mesh parameters being based on the first and second 3D mesh parameters and relative rotation and translation parameters of image sensors that captured the first and second images; determining a loss associated with the third 3D mesh parameters, the loss being based on the first and second 3D mesh parameters and the relative rotation and translation parameters; determining 3D mesh parameters based on the loss and third 3D mesh parameters.

Enhancing three-dimensional models using multi-view refinement

Systems, methods, and non-transitory media are provided for generating virtual private spaces for extended reality (XR) experiences. An example method can include initiating a virtual session for presenting virtual content and identifying, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content. The method can include outputting boundary information defining a boundary of the virtual private space, and generate at least the portion of the virtual content for the virtual private space. At least the portion of the virtual content is viewable in the virtual private space by one or more authorized users of the virtual session and is not viewable by one or more unauthorized users.

Virtual private space for extended reality

The present disclosure relates to methods and apparatus for graphics processing. Aspects of the present disclosure may identify a first content group and a second content group in a scene. Further, aspects of the present disclosure may determine whether at least a portion of the first content group occludes or potentially occludes at least a portion of the second content group. Additionally, the present disclosure may represent the first content group and the second content group based on the determination whether at least a portion of the first content group occludes or potentially occludes at least a portion of the second content group. In some aspects, the first content group may include at least some real content and the second content group includes at least some augmented content. The present disclosure may also render at least a portion of surfaces of the first content group using an occlusion material.

Methods and apparatus for handling occlusions in split rendering

Methods, systems, computer-readable media, and apparatuses for audio signal processing are presented. Some configurations include determining that first audio activity in at least one microphone signal is voice activity; determining whether the voice activity is voice activity of a participant in an application session active on a device; based at least on a result of the determining whether the voice activity is voice activity of a participant in the application session, generating an antinoise signal to cancel the first audio activity; and by a loudspeaker, producing an acoustic signal that is based on the antinoise signal. Applications relating to shared virtual spaces are described.

Audio control for extended-reality shared space

Techniques and systems are provided for providing recommendations for extended reality systems. In some examples, a system determines one or more environmental features associated with a real-world environment of an extended reality system. The system determines one or more user features associated with a user of the extended reality system. The system also outputs, based on the one or more environmental features and the one or more user features, a notification associated with at least one application supported by the extended reality system.

Recommendations for extended reality systems

Methods, systems, and devices for deep learning based head motion prediction for extended reality are described. The head pose prediction may involve training one or more layers of a machine learning network based on application data and an estimated head motion range associated with the extended reality system. The network may receive one or more bias corrected inertial measurement unit (IMU) measurements based on a sensor. The network may model a relative head pose of the user as a polynomial of time over a prediction interval based on the bias corrected IMU measurements and the trained one or more layers of the machine learning network. The network may determine a future relative head pose of the user based on the polynomial (e.g., which may be used for virtual object generation, display, etc. within an extended reality system).

Deep learning based head motion prediction for extended reality

Growing community of inventors

Vienna, Austria

Gerhard Reitmayr