Qualcomm Incorporated

Chiranjib Choudhuri

Ajit Deepak Gupte

Pushkar Gorur Sheshagiri

Anupama S

Avani Rao

Ning Bi

Pawan Kumar Baheti

Gerhard Reitmayr

Michel Adib Sarkis

Ajit Venkat Rao

Ke-Li Cheng

Chieh-Ming Kuo

Kuang-Man Huang

Chayan Sharma

Srujan Babu Nandipati

Vinay Melkote Krishnaprasad

Shilpi Sahu

Sudipto Banerjee

Tom Edward Botterill

Vinod Kumar Saini

Mario Sanchez

Thirukumaran Dechinamoorthy

Manikandan Meiyoor Velayutham

Youngmin Park

Systems and techniques are provided for an adjustable camera system for mouth tracking. An example apparatus can include a housing with an opening formed in a first side of the housing, wherein one or more surfaces of the housing are configured to engage a head of a user; and a structure including a lens configured to receive incident light, wherein the structure is configured to move from a retracted state where at least a portion of the structure is retracted into the opening in the first side of the housing, to an extended state where at least a portion of the structure that includes the lens structure extends from the first side of the housing.

Adjustable camera system

Systems and techniques are provided for generating a three-dimensional (3D) facial model. For example, a process can include obtaining at least one input image associated with a face. In some aspects, the process can include obtaining a pose for a 3D facial model associated with the face. In some examples, the process can include generating, by a machine learning model, the 3D facial model associated with the face. In some cases, one or more parameters associated with a shape component of the 3D facial model are conditioned on the pose. In some implementations, the 3D facial model is configured to vary in shape based on the pose for the 3D facial model associated with the face.

View dependent three-dimensional morphable models

Systems and techniques are described herein for using head mounted display (HMD) hinge motion to calibrate the head mounted device. For example, an device (or component of the device) can obtain, via one or more sensors, sensor measurements during rotation of a display of the HMD between a first end of a range of motion of a hinge of the display about an axis and a second end of the range of motion of the hinge about the axis; determine parameters of the HMD based on the sensor measurements and the rotation of the display between the first end of the range of motion of the hinge and the second end of the range of motion of the hinge about the axis; and determine pose information for the HMD based on the determined parameters.

Head mounted display (HMD) device calibration based on hinge motion

Systems and techniques are provided for generating a texture for a three-dimensional (3D) facial model. For example, a process can include obtaining a first frame, the first frame including a first portion of a face. In some aspects, the process can include generating a 3D facial model based on the first frame and generating a first facial feature corresponding to the first portion of the face. In some examples, the process includes obtaining a second frame, the second frame including a second portion of the face. In some cases, the second portion of the face at least partially overlaps the first portion of the face. In some examples, the process includes combining the first facial feature with the second facial feature to generate an enhanced facial feature, wherein the combining is performed to enhance an appearance of select areas of the enhanced facial feature.

Facial texture synthesis for three-dimensional morphable models

Systems and techniques are described herein for processing frames. The systems and techniques can be implemented by various types of systems, such as by an extended reality (XR) system or device. In some cases, a process can include obtaining feature information associated with a feature in a current frame, wherein the feature information is based on one or more previous frames; determining an estimated pose of the apparatus associated with the current frame; obtaining a distance associated with the feature in the current frame; and determining an estimated scale of the feature in the current frame based on the feature information associated with the feature, the estimated pose, and the distance associated with the feature.

Keypoint detection and feature descriptor computation

Systems and techniques are provided for performing user authentication. For example, a process can include obtaining a plurality of images associated with a face and a facial expression of the user, wherein each respective image of the plurality of images includes a different portion of the face. An encoder neural network can be used to generate one or more predicted three-dimensional (3D) facial modeling parameters, wherein the encoder neural network generates the one or more predicted 3D facial modeling parameters based on the plurality of images. A reference 3D facial model associated with the face and the facial expression can be obtained. An error can be determined between the one or more predicted 3D facial modeling parameters and the reference 3D facial model, and the user can be authenticated user based on the error being less than a pre-determined authentication threshold.

User authentication based on three-dimensional face modeling using partial face images

Systems and techniques are provided for determining and applying corrected poses in digital content experiences. An example method can include receiving, from one or more sensors associated with an apparatus, inertial measurements and one or more frames of a scene; based on the one or more frames and the inertial measurements, determining, via a first filter, an angular and linear motion of the apparatus and a gravity vector indicating a direction of gravitational force interacting with the apparatus; when a motion of the apparatus is below a threshold, determining, via a second filter, an updated gravity vector indicating a direction of gravitational force interacting with the apparatus; determining, based on the updated gravity vector, parameters for aligning an axis of the scene with a gravity direction in a real-world spatial frame; and aligning, using the parameters, the axis of the scene with the gravity direction in the real-world spatial frame.

Pose correction for digital content

Methods, systems, and devices for deep learning based head motion prediction for extended reality are described. The head pose prediction may involve training one or more layers of a machine learning network based on application data and an estimated head motion range associated with the extended reality system. The network may receive one or more bias corrected inertial measurement unit (IMU) measurements based on a sensor. The network may model a relative head pose of the user as a polynomial of time over a prediction interval based on the bias corrected IMU measurements and the trained one or more layers of the machine learning network. The network may determine a future relative head pose of the user based on the polynomial (e.g., which may be used for virtual object generation, display, etc. within an extended reality system).

Deep learning based head motion prediction for extended reality

Systems, methods, and computer-readable media are provided for distributed tracking and mapping for extended reality experiences. An example method can include computing, at a device, a pose of the device at a future time, the future time being determined based on a communication latency between the device and a mapping backend system; sending, to the mapping backend system, the pose of the device; receiving, from the mapping backend system, a map slice including map points corresponding to a scene associated with the device, the map slice being generated based on the pose of the device, wherein the map points correspond to the predicted pose; and computing an updated pose of the device based on the map slice.

Distributed pose estimation

Systems, methods, and computer-readable media are provided for immersive extended reality experiences on mobile platforms. In some examples, a method can include obtaining sensor measurements from one or more sensors on a mobile platform and/or a device associated with a user in the mobile platform, the sensor measurements including motion parameters associated with the mobile platform and the user; identifying features of the mobile platform and an environment outside of the mobile platform; tracking, using the sensor measurements, a first pose of the mobile platform relative to the environment outside of the mobile platform; tracking, using the sensor measurements, a second pose of the user relative to at least one of the features of the mobile platform; and tracking, based on the first pose and the second pose, a third pose of the user relative to at least one of the features of the environment outside of the mobile platform.

Systems and methods for providing immersive extended reality experiences on moving platforms

Techniques are described in which a device is configured to determine an overlap region between a first image and a second image, determine a first histogram based on color data included in the first image that corresponds to the overlap region, and determine a second histogram based on color data included in the second image that corresponds to the overlap region. The processor is further configured to determine, based on the first and second histograms, a mapping function that substantially maps the second histogram to the first histogram and apply the mapping function to the second image to generate a normalized second image with respect to the first image.

Color normalization for a multi-camera system

A method for stitching images by an electronic device is described. The method includes obtaining at least two images. The method also includes selecting a stitching scheme from a set of stitching schemes based on one or more content measures of the at least two images. The set of stitching schemes includes a first stitching scheme, a second stitching scheme, and a third stitching scheme. The method further includes stitching the at least two images based on a selected stitching scheme.

Growing community of inventors

Bengaluru, India

Chiranjib Choudhuri