Qualcomm Incorporated

Murali Chari

Fuyun Ling

Rajiv Vijayan

Raghuraman Krishnamoorthi

Arvind Ramanandan

Gordon Kent Walker

Christopher Brunner

Mahesh Ramachandran

Ashok Mantravadi

Krishna Kiran Mukkavilli

Michael Mao Wang

Ashwin Deepak Swaminathan

Serafin Diaz Spindola

Durk L Van Veen

Abhishek Tyagi

Pawan Kumar Baheti

Seong Taek Chung

Mark Leo Moeglein

Bruce Collins

Bojan Vrcelj

Thomas Sun

Avdhut Joshi

Piyush Sharma

Aamod Dinkar Khandekar

Sanjiv Nanda

Avneesh Agrawal

Manoj Madhukar Deshpande

Vidya Narayanan

Ranjith Subramanian Jayaram

Linbo Li

Jiming Guo

Prasanna Kannan

Rajeev Krishnamurthi

Yanghai Tsin

Devarshi P Shah

Vinay Murthy

Rinat Burdo

Bolan Jiang

Shusheel Gautam

George Allen Rothrock

Michael M Fan

Pankaj V Rahate

Pankaj Jain

Pavel A Seliverstov

Nilabh Khare

Anil K Wadhwani

Reynaldo W Newman

Chong U Lee

Hui Chao

Yuriy Reznik

John Hyunchul Hong

Daniel Wagner

Tao Tian

Urs Niesen

Ming-Chang Tsai

Alok Kumar Gupta

Jai N Subrahmanyam

Onur Canturk Hamsici

Jinxia Bai

Anand D Subramaniam

Sundeep Vaddadi

Dhinakar Radhakrishnan

Seyed Hesameddin Najafi Shoushtari

Paulo Ricardo Dos Santos Mendonca

Dheeraj Ahuja

Daniel Knoblauch

Prasun Choudhury

Yiming Chen

Raghuraman Krisnamoorthi

John Steven Lima

Jose Jeronimo Moreira Rodrigues

Disclosed are techniques for performing object detection and tracking. In some implementations, a process for performing object detection and tracking is provided. The process can include steps for obtaining, at a tracking object, an image comprising a target object, obtaining, at the tracking object, a first set of messages associated with the target object, determining a bounding box for the target object in the image based on the first set of messages associated with the target object, and extracting a sub-image from the image. In some approaches, the process can further include steps for detecting, using an object detection model, a location of the target object within the sub-image. Systems and machine-readable media are also provided.

Objection detection using images and message information

An electronic device is described. The electronic device includes a memory and a processor in communication with the memory. The memory is configured to store precalibration data for a camera mounted on a vehicle, the precalibration data including a camera height determined relative to a road plane the vehicle is configured to contact during operation. The processor is configured to receive a plurality of images. The processor is also configured to classify one or more features in the plurality of images as road features based on the precalibration data.

Systems and methods for classifying road features

A mobile device determines a vision based pose using images captured by a camera and determines a sensor based pose using data from inertial sensors, such as accelerometers and gyroscopes. The vision based pose and sensor based pose are used separately in a visualization application, which displays separate graphics for the different poses. For example, the visualization application may be used to calibrate the inertial sensors, where the visualization application displays a graphic based on the vision based pose and a graphic based on the sensor based pose and prompts a user to move the mobile device in a specific direction with the displayed graphics to accelerate convergence of the calibration of the inertial sensors. Alternatively, the visualization application may be a motion based game or a photography application that displays separate graphics using the vision based pose and the sensor based pose.

Display of separate computer vision based pose and inertial sensor based pose

A method performed by an electronic device is described. The method includes determining a predicted velocity relative to Earth corresponding to a first epoch using a camera and an inertial measurement unit (IMU). The method also includes determining, using a Global Positioning System (GPS) receiver, a GPS velocity relative to Earth. The method further includes determining a difference vector between the predicted velocity and the GPS velocity. The method additionally includes refining a bias estimate and a scale factor estimate of IMU measurements proportional to the difference vector. The method also includes refining a misalignment estimate between the camera and the IMU based on the difference vector. The method further includes providing pose information based on the refined bias estimate, the refined scale factor, and the refined misalignment estimate.

Systems and methods for using a global positioning system velocity in visual-inertial odometry

A method for visual inertial odometry (VIO)-aided global positioning is described. The method includes updating an extended Kalman filter (EKF) state including a current pose and a sliding window of multiple prior poses. The sliding window includes poses at a number of most recent global positioning system (GPS) time epochs. Updating the EKF includes updating an EKF covariance matrix for the prior poses and the current pose in the EKF state. The method also includes determining, at a GPS epoch, a relative displacement between each of the updated prior poses and the current pose. The method further includes determining an error covariance of each of the relative displacements based on cross-covariances between each of the updated prior poses and the current pose in the EKF covariance matrix. The method additionally includes using the relative displacements and the error covariances to fuse pseudorange measurements taken over multiple epochs.

Systems and methods for using a sliding window of global positioning epochs in visual-inertial odometry

An apparatus and method for a framework exposing an API (application programming interface) to web-based server applications on the internet or in the cloud is presented. The API allows server applications to retrieve sensor data from a mobile device via a low-power sensor core processor on a mobile device. This API eliminates effort and cost associated with developing and promoting a new mobile device client application. The API framework includes APIs that web-based application may use to fetch sensor data from one or more particular sensors on the mobile device.

Sensor API framework for cloud based applications

A first map comprising local features and 3D locations of the local features is generated, the local features comprising visible features in a current image and a corresponding set of covisible features. A second map comprising prior features and 3D locations of the prior features may be determined, where each prior feature: was first imaged at a time prior to the first imaging of any of the local features, and lies within a threshold distance of at least one local feature. A first subset comprising previously imaged local features in the first map and a corresponding second subset of the prior features in the second map is determined by comparing the first and second maps, where each local feature in the first subset corresponds to a distinct prior feature in the second subset. A transformation mapping a subset of local features to a subset of prior features is determined.

Geometric matching in visual navigation systems

Embodiments disclosed pertain to the use of user equipment (UE) for the generation of a 3D exterior envelope of a structure based on captured images and a measurement set associated with each captured image. In some embodiments, a sequence of exterior images of a structure is captured and a corresponding measurement set comprising Inertial Measurement Unit (IMU) measurements, wireless measurements (including Global Navigation Satellite (GNSS) measurements) and/or other non-wireless sensor measurements may be obtained concurrently. A closed-loop trajectory of the UE in global coordinates may be determined and a 3D structural envelope of the structure may be obtained based on the closed loop trajectory and feature points in a subset of images selected from the sequence of exterior images of the structure.

Exterior hybrid photo mapping

An accelerometer in a mobile device is calibrated by taking multiple measurements of acceleration vectors when the mobile device is held stationary at different orientations with respect to a plane normal. A circle is calculated that fits respective tips of measured acceleration vectors in the accelerometer coordinate system. The radius of the circle and the lengths of the measured acceleration vectors are used to calculate a rotation angle for aligning the accelerometer coordinate system with the mobile device surface. A gyroscope in the mobile device is calibrated by taking multiple measurements of a rotation axis when the mobile device is rotated at different rates with respect to the rotation axis. A line is calculated that fits the measurements. The angle between the line and an axis of the gyroscope coordinate system is used to align the gyroscope coordinate system with the mobile device surface.

Method for aligning a mobile device surface with the coordinate system of a sensor

Techniques for multiplexing and transmitting multiple data streams are described. Transmission of the multiple data streams occurs in 'super-frames'. Each super-frame has a predetermined time duration and is further divided into multiple (e.g., four) frames. Each data block for each data stream is outer encoded to generate a corresponding code block. Each code block is partitioned into multiple subblocks, and each data packet in each code block is inner encoded and modulated to generate modulation symbols for the packet. The multiple subblocks for each code block are transmitted in the multiple frames of the same super-frame, one subblock per frame. Each data stream is allocated a number of transmission units in each super-frame and is assigned specific transmission units to achieve efficient packing. A wireless device can select and receive individual data streams.

Multiplexing and transmission of multiple data streams in a wireless multi-carrier communication system

A mobile platform efficiently processes image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires image data, and determines whether there is a trigger event to transmit the image data to the server. The trigger event may be a change in the image data relative to previously acquired image data, e.g., a scene change in an image. When a change is present, the image data may be transmitted to the server for processing. The server processes the image data and returns information related to the image data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Adaptable framework for cloud assisted augmented reality

Methods, systems, computer-readable media, and apparatuses for fast cost aggregation for dense stereo matching are presented. One example method includes the steps of receiving first and second images of a scene; rectifying the images; computing a cost volume based on the first and second images; subsampling the cost volume to generate a subsampled cost volume; for each pixel, p, in the subsampled cost volume, determining one or more local extrema in the subsampled cost volume for each neighboring pixel, q, within a window centered on the pixel, p; for each pixel, p, performing cost aggregation using the one or more local extrema; performing cross checking to identify matching pixels; and responsive to identifying unmatched pixels, performing gap-filling for the unmatched pixels to generate a disparity map; and generate and storing a depth map from the disparity map.

Fast cost aggregation for dense stereo matching

A mobile platform efficiently processes sensor data, including image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires sensor data, such as image data, and determines whether there is a trigger event to transmit the sensor data to the server. The trigger event may be a change in the sensor data relative to previously acquired sensor data, e.g., a scene change in an image. When a change is present, the sensor data may be transmitted to the server for processing. The server processes the sensor data and returns information related to the sensor data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

A venue-cast system and method for providing and receiving venue level transmissions and services, including discovery of a venue specific transmission by receiving an overhead signal from a non-venue network, extracting information for receiving the venue specific transmission from the overhead signal, and tuning to receive the venue specific transmission based on the extracted information. The venue level transmission may be provided and received in a manner that does not prevent an access terminal from receiving a local area or wide area transmission.

Apparatus and methods of providing and receiving venue level transmissions and services

Growing community of inventors

San Diego, CA, United States of America

Murali Chari