Qualcomm Incorporated

Bolan Jiang

Ashwin Deepak Swaminathan

Serafin Diaz Spindola

Dheeraj Ahuja

Charles Wheeler Sweet, Iii

Prasun Choudhury

Vidya Narayanan

Murali Chari

Pawan Kumar Baheti

Piyush Sharma

Ning Bi

Yingyong Qi

Shuxue Quan

Chienchung Chang

Christopher Brunner

Feng Guo

Seyed Hesameddin Najafi Shoushtari

Xin Li

Yunke Pan

Sheng Yi

Iii Charles Wheeler Sweet

Murali R Chari

Provided are systems, methods, and computer-readable medium for operating a neural network. In various implementations, the neural network can receive an input image that includes an object to be identified. The neural network can generate a plurality of initial feature maps using a convolution layers, wherein a first initial feature maps is generated using the input image. The neural network can generate an up-sampled feature map using a de-convolution layer that takes an initial feature map as input, where the up-sampled feature map has a same resolution as the previous initial feature map. The neural network can combine the up-sampled feature map and the previous initial feature map, and use the combined feature map to more accurate identify the object.

Combining convolution and deconvolution for object detection

An apparatus includes a first sensor configured to generate first sensor data. The first sensor data is related to an occupant of a vehicle. The apparatus further includes a depth sensor and a processor. The depth sensor is configured to generate data corresponding to a volume associated with at least a portion of the occupant. The processor is configured to receive the first sensor data and to activate the depth sensor based on the first sensor data.

System and method of depth sensor activation

A mobile platform efficiently processes image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires image data, and determines whether there is a trigger event to transmit the image data to the server. The trigger event may be a change in the image data relative to previously acquired image data, e.g., a scene change in an image. When a change is present, the image data may be transmitted to the server for processing. The server processes the image data and returns information related to the image data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Adaptable framework for cloud assisted augmented reality

A vision based tracking system in a mobile platform tracks objects using groups of detected lines. The tracking system detects lines in a captured image of the object to be tracked. Groups of lines are formed from the detected lines. The groups of lines may be formed by computing intersection points of the detected lines and using intersection points to identified connected lines, where the groups of lines are formed using connected lines. A graph of the detected lines may be constructed and intersection points identified. Interesting subgraphs are generated using the connections and the group of lines is formed with the interesting subgraphs. Once the groups of lines are formed, the groups of lines are used to track the object, e.g., by comparing the groups of lines in a current image of the object to groups of lines in a previous image of the object.

Line tracking with automatic model initialization by graph matching and cycle detection

A multi-user augmented reality (AR) system operates without a previously acquired common reference by generating a reference image on the fly. The reference image is produced by capturing at least two images of a planar object and using the images to determine a pose (position and orientation) of a first mobile platform with respect to the planar object. Based on the orientation of the mobile platform, an image of the planar object, which may be one of the initial images or a subsequently captured image, is warped to produce the reference image of a front view of the planar object. The reference image may be produced by the mobile platform or by, e.g., a server. Other mobile platforms may determine their pose with respect to the planar object using the reference image to perform a multi-user augmented reality application.

Online reference generation and tracking for multi-user augmented reality

A mobile platform efficiently processes sensor data, including image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires sensor data, such as image data, and determines whether there is a trigger event to transmit the sensor data to the server. The trigger event may be a change in the sensor data relative to previously acquired sensor data, e.g., a scene change in an image. When a change is present, the sensor data may be transmitted to the server for processing. The server processes the sensor data and returns information related to the sensor data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

A homography between two captured images of a planar object is decomposed into at least one possible solution, and typically at least two ambiguous solutions. The ambiguity between the two solutions is removed, or a single solution validated, using measurements from orientation sensors. The measurements from orientation sensors may be used by comparing at least one of the yaw, pitch, and/or roll angles derived from a relative rotation matrix for the one or more solutions to a corresponding at least one of the yaw, pitch, and/or roll angles derived from the measurements from the orientation sensors.

Resolving homography decomposition ambiguity based on orientation sensors

A reference patch of an unknown environment is generated on the fly for positioning and tracking. The reference patch is generated using a captured image of a planar object with two perpendicular sets of parallel lines. The planar object is detected in the image and axes of the world coordinate system are defined using the vanishing points for the two sets of parallel lines. The camera rotation is recovered based on the defined axes, and the reference patch of at least a portion of the image of the planar object is generated using the recovered camera rotation. The reference patch can then be used for vision based detection and tracking. The planar object may be detected in the image as sets of parallel lines or as a rectangle.

Online reference patch generation and pose estimation for augmented reality

A mobile station determines its orientation using an image of an object produced by the mobile station and a top view of that object obtained from an online server. The mobile station image is analyzed to identify lines on the object and to determine the direction of the lines with respect to the mobile station. The top view image, which may be a satellite image, is also analyzed to identify lines on the object that correspond to the lines identified in the mobile station image. The direction of the lines in the top view image are compared to the direction of lines in the mobile station image and based on their relative orientation the orientation of the mobile station may be determined. For example, the difference between the preliminary and corrected orientations may be stored as a calibration factor and used to correct subsequent orientation measurements from orientation sensors.

Growing community of inventors

San Diego, CA, United States of America

Bolan Jiang