Qualcomm Incorporated

Yan Deng

Michel Adib Sarkis

Ning Bi

Yingyong Qi

Nikolai Konrad Leung

Chieh-Ming Kuo

Ze Zhang

Techniques and systems are provided for image processing. For instance, a process can include obtaining, from one or more image sensors, a first image of an environment; determining semantic labels for a plurality of pixels of the first image based on whether each pixel of the plurality of pixels is associated with an object in the first image to generate a semantically segmented image; applying a two-dimensional line representation of the first image to the semantically segmented image to generate a labeled 2D line representation; back-projecting the labeled 2D line representation to three dimensions to generate a line representation model; fusing the line representation model and the semantically segmented image to generate a labeled line representation model; and outputting the labeled line representation model.

Edge-based geometry representation for cross layer applications

Systems and techniques are described herein for generating one or more three-dimensional models. For instance, a method for generating one or more three-dimensional models is provided. The method may include obtaining a plurality of images of an object; obtaining a plurality of segmentation masks associated with the plurality of images, each segmentation mask of the plurality of segmentation masks including at least one label indicative of at least one segment of the object in a respective image of the plurality of images; training, using the plurality of images and the plurality of segmentation masks, a machine-learning model to generate one or more semantically-labeled three-dimensional models of the object; and generating using the trained machine-learning model, a semantically-labeled three-dimensional model of the object, the semantically-labeled three-dimensional model of the object including at least one label indicative of the at least one segment of the object.

Generating semantically-labelled three-dimensional models

Systems and techniques are provided for performing video-based activity recognition. For example, a process can include generating a three-dimensional (3D) model of a first portion of an object based on one or more frames depicting the object. The process can also include generating a mask for the one or more frames, the mask including an indication of one or more regions of the object. The process can further include generating a 3D base model based on the 3D model of the first portion of the object and the mask, the 3D base model representing the first portion of the object and a second portion of the object. The process can include generating, based on the mask and the 3D base model, a 3D model of the second portion of the object.

Object reconstruction using media data

Systems and techniques are provided for performing video-based activity recognition. For example, a process can include generating a three-dimensional (3D) model of a first portion of an object based on one or more frames depicting the object. The process can also include generating a mask for the one or more frames, the mask including an indication of one or more of regions of the object. The process can further include generating a 3D base model based on the 3D model of the first portion of the object and the mask, the 3D base model representing the first portion of the object and a second portion of the object. The process can include generating, based on the mask and the 3D base model, a 3D model of the second portion of the object.

A method performed by an electronic device is described. The method includes receiving first optical data and first depth data corresponding to a first frame. The method also includes registering the first depth data to a first canonical model. The method further includes fitting a three-dimensional (3D) morphable model to the first optical data. The method additionally includes registering the 3D morphable model to a second canonical model. The method also includes producing a 3D object reconstruction based on the registered first depth data and the registered 3D morphable model.

Systems and methods for reconstructing a three-dimensional object

A method performed by an electronic device is described. The method includes receiving a set of frames. The set of frames describes a moving three-dimensional (3D) object. The method also includes registering the set of frames based on a canonical model. The canonical model includes geometric information and optical information. The method additionally includes fusing frame information of each frame to the canonical model based on the registration. The method further includes reconstructing the 3D object based on the canonical model.

Systems and methods for reconstructing a moving three-dimensional object

A method performed by an electronic device is described. The method includes incrementally adding a current node to a graph. The method also includes incrementally determining a respective adaptive edge threshold for each candidate edge between the current node and one or more candidate neighbor nodes. The method further includes determining whether to accept or reject each candidate edge based on each respective adaptive edge threshold. The method additionally includes performing refining based on the graph to produce refined data. The method also includes producing a three-dimensional (3D) model based on the refined data.

Systems and methods for outlier edge rejection

A method is described. The method includes determining normalized radiance of an image sequence based on a camera response function (CRF). The method also includes determining one or more reliability images of the image sequence based on a reliability function corresponding to the CRF. The method further includes extracting features based on the normalized radiance of the image sequence. The method additionally includes optimizing a model based on the extracted features and the reliability images.

Growing community of inventors

San Diego, CA, United States of America

Yan Deng