Apple Inc.

Qualcomm Incorporated

Seyed Hesameddin Najafi Shoushtari

Yanghai Tsin

Kuen-Han Lin

Martin Paul Grunthaner

Murali Chari

Jiaying Wu

Ashwin Deepak Swaminathan

Nicolas P Bonnier

Regan Blythe Towal

Bolan Jiang

Joshua M Susskind

Aliakbar Aghamohammadi

Kiyoung Kim

Dheeraj Ahuja

Shih-Yu Sun

Chen Huang

William Mathews Riedel

Sheng Yi

Frankie Lu

Feature descriptor matching is reformulated into a graph-matching problem. Keypoints from a query image and a reference image are initially matched and filtered based on the match. For a given keypoint, a feature graph is constructed based on neighboring keypoints surrounding the given keypoint. The feature graph is compared to a corresponding feature graph of a reference image for the matched keypoint. Relocalization data is obtained based on the comparison.

Contextual matching

Methods and apparatus for glint-assisted gaze tracking in a VR/AR head-mounted display (HMD). Images of a user's eyes captured by gaze tracking cameras may be analyzed to detect glints (reflections on the cornea of light sources that illuminate the user's eyes) and the pupil. The glints are matched to particular ones of the light sources. The glint-light source matches are used to determine the cornea center of the eye, and the pupil center is determined. The optical axis of the eye is reconstructed from the cornea center and the pupil center, and the visual axis is then reconstructed from the optical axis and a 3D model of the user's eye. The point of gaze on the display is then determined based on the visual axis and a 3D model of the HMD.

Glint-assisted gaze tracker

An electronic device may have a display and a gaze tracking system. Control circuitry in the electronic device can produce a saliency map in which items of visual interest are identified among content that has been displayed on a display in the electronic device. The saliency map may identify items such as selectable buttons, text, and other items of visual interest. User input such as mouse clicks, voice commands, and other commands may be used by the control circuitry in identifying when a user is gazing on particular items within the displayed content. Information on a user's actual on-screen point of gaze that is inferred using the saliency map information and user input can be compared to measured eye position information from the gaze tracking system to calibrate the gaze tracking system during normal operation of the electronic device.

Electronic device with gaze tracking system

A method for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning includes recognizing and localizing an object in a two-dimensional (2D) image. The method also includes computing 3D depth maps for the localized object. A 3D object map is constructed from the depth maps. A sampling based structure is grown around the 3D object map and a cost is assigned to each edge of the sampling based structure. The sampling based structure may be searched to determine a lowest cost sequence of edges that may, in turn be used to guide the robot.

Object-focused active three-dimensional reconstruction

Methods, systems, computer-readable media, and apparatuses for fast cost aggregation for dense stereo matching are presented. One example method includes the steps of receiving first and second images of a scene; rectifying the images; computing a cost volume based on the first and second images; subsampling the cost volume to generate a subsampled cost volume; for each pixel, p, in the subsampled cost volume, determining one or more local extrema in the subsampled cost volume for each neighboring pixel, q, within a window centered on the pixel, p; for each pixel, p, performing cost aggregation using the one or more local extrema; performing cross checking to identify matching pixels; and responsive to identifying unmatched pixels, performing gap-filling for the unmatched pixels to generate a disparity map; and generate and storing a depth map from the disparity map.

Fast cost aggregation for dense stereo matching

A vision based tracking system in a mobile platform tracks objects using groups of detected lines. The tracking system detects lines in a captured image of the object to be tracked. Groups of lines are formed from the detected lines. The groups of lines may be formed by computing intersection points of the detected lines and using intersection points to identified connected lines, where the groups of lines are formed using connected lines. A graph of the detected lines may be constructed and intersection points identified. Interesting subgraphs are generated using the connections and the group of lines is formed with the interesting subgraphs. Once the groups of lines are formed, the groups of lines are used to track the object, e.g., by comparing the groups of lines in a current image of the object to groups of lines in a previous image of the object.

Line tracking with automatic model initialization by graph matching and cycle detection

Disclosed embodiments pertain to apparatus, systems, and methods for robust feature based tracking. In some embodiments, a score may be computed for a camera captured current image comprising a target object. The score may be based on one or more metrics determined from a comparison of features in the current image and a prior image captured by the camera. The comparison may be based on an estimated camera pose for the current image. In some embodiments, one of a point based, an edge based, or a combined point and edge based feature correspondence method may be selected based on a comparison of the score with a point threshold and/or a line threshold, the point and line thresholds being obtained from a model of the target. The camera pose may be refined by establishing feature correspondences using the selected method between the current image and a model image.

Robust tracking using point and line features

Growing community of inventors

San Diego, CA, United States of America

Seyed Hesameddin Najafi Shoushtari