Qualcomm Incorporated

Sony Corporation

Michel Adib Sarkis

Ning Bi

Yingyong Qi

Ke-Li Cheng

Yan Deng

Sairam Sundaresan

Kuang-Man Huang

Chieh-Ming Kuo

Amichai Sanderovich

Bijan Forutanpour

Evyatar Hemo

Matthew Fischler

Sharad Deepak Sambhwani

Ajit Deepak Gupte

Gerhard Reitmayr

Mainak Biswas

Scott David Beith

Evgeny Levitan

Chiranjib Choudhuri

Eran Hof

Magdi Mohamed

Anupama S

Avani Rao

Kuen-Han Lin

Jeremiah Golston

Mohammad Faroq Salama

Lei Wang

Nikolai Konrad Leung

Stephane Pierre Villette

Suzana Arellano

Xuan Zou

Mithun Kumar Ranganath

Wei-Lun Chang

Peng Liu

Min-Hui Lin

Ze Zhang

Arthur Pajak

Systems and techniques are provided for generating a three-dimensional (3D) facial model. For example, a process can include obtaining at least one input image associated with a face. In some aspects, the process can include obtaining a pose for a 3D facial model associated with the face. In some examples, the process can include generating, by a machine learning model, the 3D facial model associated with the face. In some cases, one or more parameters associated with a shape component of the 3D facial model are conditioned on the pose. In some implementations, the 3D facial model is configured to vary in shape based on the pose for the 3D facial model associated with the face.

View dependent three-dimensional morphable models

Techniques and systems are provided for image processing. For instance, a process can include obtaining, from one or more image sensors, a first image of an environment; determining semantic labels for a plurality of pixels of the first image based on whether each pixel of the plurality of pixels is associated with an object in the first image to generate a semantically segmented image; applying a two-dimensional line representation of the first image to the semantically segmented image to generate a labeled 2D line representation; back-projecting the labeled 2D line representation to three dimensions to generate a line representation model; fusing the line representation model and the semantically segmented image to generate a labeled line representation model; and outputting the labeled line representation model.

Edge-based geometry representation for cross layer applications

Systems and techniques are provided for generating a texture for a three-dimensional (3D) facial model. For example, a process can include obtaining a first frame, the first frame including a first portion of a face. In some aspects, the process can include generating a 3D facial model based on the first frame and generating a first facial feature corresponding to the first portion of the face. In some examples, the process includes obtaining a second frame, the second frame including a second portion of the face. In some cases, the second portion of the face at least partially overlaps the first portion of the face. In some examples, the process includes combining the first facial feature with the second facial feature to generate an enhanced facial feature, wherein the combining is performed to enhance an appearance of select areas of the enhanced facial feature.

Facial texture synthesis for three-dimensional morphable models

Systems and techniques are provided for deep learning model optimizations. An example process can include generating, based on processing data using a neural network model, a plurality of prediction outputs associated with a plurality of output channels corresponding to a multi-channel prediction target; determining that a prediction output from the plurality of prediction outputs has a value that is outside of a quantization range and one or more remaining prediction outputs from the plurality of prediction outputs have a respective value that is within the quantization range; clamping the prediction output based on the quantization range; and generating a single channel output based on the clamped prediction output and the one or more remaining prediction outputs.

Deep learning model for high resolution predictions

A device includes a memory and one or more processors configured to process sensor data to determine a semantical context associated with the sensor data. The one or more processors are also configured to generate adjusted face data based on the determined semantical context and face data. The adjusted face data includes an avatar facial expression that corresponds to the semantical context.

Avatar facial expressions based on semantical context

Systems and techniques are described herein for generating one or more three-dimensional models. For instance, a method for generating one or more three-dimensional models is provided. The method may include obtaining a plurality of images of an object; obtaining a plurality of segmentation masks associated with the plurality of images, each segmentation mask of the plurality of segmentation masks including at least one label indicative of at least one segment of the object in a respective image of the plurality of images; training, using the plurality of images and the plurality of segmentation masks, a machine-learning model to generate one or more semantically-labeled three-dimensional models of the object; and generating using the trained machine-learning model, a semantically-labeled three-dimensional model of the object, the semantically-labeled three-dimensional model of the object including at least one label indicative of the at least one segment of the object.

Generating semantically-labelled three-dimensional models

Disclosed are techniques for processing image data. In some aspects, a three-dimensional model can be determined corresponding to an object in an input image. Based on the three-dimensional model, an estimated focal length can be determined that corresponds to the input image. An estimated depth associated with the object in the input image can be calculated based on the estimated focal length and an input image focal length.

Image processing framework for performing object depth estimation

Methods, systems, and apparatuses are provided to automatically reconstruct an image, such as a 3D image. For example, a computing device may obtain an image, and may apply a first trained machine learning process to the image to generate coefficient values characterizing the image in a plurality of dimensions. Further, the computing device may generate a mesh based on the coefficient values. The computing device may apply a second trained machine learning process to the coefficient values and the image to generate a displacement map. Based on the mesh and the displacement map, the computing device may generate output data characterizing an aligned mesh. The computing device may store the output data within a data repository. In some examples, the computing device provides the output data for display.

Apparatus and methods for image reconstruction using machine learning processes

Techniques are provided for generating three-dimensional models of objects from one or more images or frames. For example, at least one frame of an object in a scene can be obtained. A portion of the object is positioned on a plane in the at least one frame. The plane can be detected in the at least one frame and, based on the detected plane, the object can be segmented from the plane in the at least one frame. A three-dimensional (3D) model of the object can be generated based on segmenting the object from the plane. A refined mesh can be generated for a portion of the 3D model corresponding to the portion of the object positioned on the plane.

Object scanning using planar segmentation

A method performed by an electronic device is described. The method includes obtaining sensor data corresponding to multiple occupants from an interior of a vehicle. The method also includes obtaining, by a processor, at least one occupant status for at least one of the occupants based on a first portion of the sensor data. The method further includes identifying, by the processor, at least one vehicle operation in response to the at least one occupant status. The method additionally includes determining, by the processor, based at least on a second portion of the sensor data, whether to perform the at least one vehicle operation. The method also includes performing the at least one vehicle operation in a case that it is determined to perform the at least one vehicle operation.

Systems and methods for operating a vehicle based on sensor data

Systems and techniques are provided for performing video-based activity recognition. For example, a process can include generating a three-dimensional (3D) model of a first portion of an object based on one or more frames depicting the object. The process can also include generating a mask for the one or more frames, the mask including an indication of one or more regions of the object. The process can further include generating a 3D base model based on the 3D model of the first portion of the object and the mask, the 3D base model representing the first portion of the object and a second portion of the object. The process can include generating, based on the mask and the 3D base model, a 3D model of the second portion of the object.

Object reconstruction using media data

Systems and techniques are provided for generating one or more models. For example, a process can include obtaining a plurality of input images corresponding to faces of one or more people during a training interval. The process can include determining a value of the coefficient representing at least the portion of the facial expression for each of the plurality of input images during the training interval. The process can include determining, from the determined values of the coefficient representing at least the portion of the facial expression for each of the plurality of input images during the training interval, an extremum value of the coefficient representing at least the portion of the facial expression during the training interval. The process can include generating an updated bounding value for the coefficient representing at least the portion of the facial expression based on the initial bounding value and the extremum value.

Adaptive bounding for three-dimensional morphable models

Systems and techniques are provided for facial expression recognition. In some examples, a system receives an image frame corresponding to a face of a person. The system also determines, based on a three-dimensional model of the face, landmark feature information associated with landmark features of the face. The system then inputs, to at least one layer of a neural network trained for facial expression recognition, the image frame and the landmark feature information. The system further determines, using the neural network, a facial expression associated with the face.

Facial expression recognition

Growing community of inventors

San Diego, CA, United States of America

Michel Adib Sarkis