Aquifi, Inc.

Apple Inc.

Imimtek, Inc.

Michele Stoppa

Carlo Dal Mutto

Giuliano Pasqualotto

Britta Silke Hummel

Giridhar Murali

Abbas Rafii

Pietro Salvagnini

Ahmed Tashrif Kamal

Teresa Mao

John B Morrell

Seung Wook Kim

Francesco Peruch

Senem E Emgin

Aryan Hazeghi

Giulio Marin

Waleed Abdulla

Sree Harsha Kalli

Amir Hossein Khalili

Henning Tjaden

Raffi A Bedikian

Mohamed Selim Ben Himane

Paolo Di Febbo

Anthony Ghannoum

Oliver Thomas Ruepp

Jai Prakash

Kiranjit Dhaliwal

David Demirdjian

Johan Hedborg

Rahul Raguram

A physical keyboard can be used to collect user input in a typing mode or in a tracking mode. To use a tracking mode, first movement data is detected for a hand of a user in relation to a physical keyboard at a first location. A determination is made that the first movement data is associated with a tracking movement. In response to determining that the movement type is associated with the tracking movement, a tracking mode is initiated. User input is provided based on the movement data and in accordance with the tracking mode. Contact data and non-contact data is used to determine a user intent, and a user instruction is processed based on the user intent.

System for improving user input recognition on touch surfaces

A tracked device may be used in an extended reality system in coordination with a tracking device. The tracked device may be ordinarily difficult to track, for example due to changing appearances or relatively small surface areas of unchanging features, as may be the case with an electronic device with a relatively large display surrounded by a thin physical outer boundary. In these cases, the tracked device may periodically present an image to the tracking device that the tracking device stores as an indication to permit tracking of a known, unchanging feature despite the image not being presented continuously on the display of the tracked device. The image may include a static image, designated tracking data overlaid on an image frame otherwise scheduled for presentation, or extracted image features from the image frame otherwise scheduled for presentation. Additional power saving methods and known marker generation methods are also described.

Display tracking systems and methods

Methods, systems, and computer readable media for providing personalized saliency models, e.g., for use in mixed reality environments, are disclosed herein, comprising: obtaining, from a server, a first saliency model for the characterization of captured images, wherein the first saliency model represents a global saliency model; capturing a first plurality of images by a first device; obtaining information indicative of a reaction of a first user of the first device to the capture of one or more images of the first plurality images; updating the first saliency model based, at least in part, on the obtained information to form a personalized, second saliency model; and transmitting at least a portion of the second saliency model to the server for inclusion into the global saliency model. In some embodiments, a user's personalized (i.e., updated) saliency model may be used to modify one or more characteristics of at least one subsequently captured image.

Systems and methods for providing personalized saliency models

In one implementation, a method of estimating the heading of a device is performed by the device including a processor, non-transitory memory, and an image sensor. The method includes determining a geographic location of the device. The method includes capturing, using the image sensor, an image at the geographic location. The method includes detecting one or more lines within the image. The method includes determining a heading of the device based on the one or more lines and the geographic location.

Method and device for heading estimation

A method for a computing device to recalibrate a multiple camera system includes collecting calibration data from one or more pictures captured by the multiple camera system of the computing device. The multiple camera system includes two or more cameras that are each physically separated by a distance from one another. The method further includes detecting decalibration of the camera system. The method further includes, when the camera system is decalibrated, generating recalibration parameters based on the calibration data. The method further includes determining whether the recalibration parameters are valid parameters and, when they are, updating the multiple camera system based on the recalibration parameters.

Multiple camera system with auto recalibration

A method for capturing a depth map includes: controlling a plurality of cameras to capture, concurrently, a plurality of first images during a first exposure interval, each of the cameras concurrently capturing a corresponding one of the first images, the cameras having overlapping fields of view; controlling a projection source to emit light at a first illumination level during the first exposure interval; controlling the cameras to capture, concurrently, a plurality of second images during a second exposure interval, each of the cameras concurrently capturing a corresponding one of the second images; controlling the projection source to emit light at a second illumination level during the second exposure interval, the second illumination level being different from the first illumination level; combining the first images with the second images to generate a depth map; and outputting the depth map.

System and method for high dynamic range depth capture using multiple cameras

A pattern projection system includes a coherent light source, a repositionable DOE disposed to receive coherent light from said coherent light source and disposed to output at least one pattern of projectable light onto a scene to be imaged by an (x,y) two-dimensional optical acquisition system. Coherent light speckle artifacts in the projected pattern are reduced by rapidly controllably repositioning the DOE or the entire pattern projection system. Different projectable patterns are selected from a set of M patterns that are related to each other by a translation and/or rotation operation in two-dimensional cosine space. A resultant (x,y,z) depth map has improved quality and robustness due to projection of the selected patterns. Three-dimensional (x,y,z) depth data obtained from two-dimensional imaged data including despeckling is higher quality data than if projected patterns without despeckling were used.

Systems and methods for compact space-time stereo three-dimensional depth sensing

A method for detecting decalibration of a depth camera system including a first, second, and third cameras having overlapping fields of view in a direction includes: detecting a feature in a first image captured by the first camera; detecting the feature in a second image captured by the second camera; detecting the feature in a third image captured by the third camera, the third camera being non-collinear with the first and second cameras; identifying a first conjugate epipolar line in the second image in accordance with a detected location of the feature in the first image and calibration parameters; identifying a second conjugate epipolar line in the second image in accordance with a detected location of the feature in the third image and the calibration parameters; and calculating a difference between a detected location of the feature in the second image and the first and second conjugate epipolar lines.

Depth perceptive trinocular camera system

A method for operating a real-time gesture based interactive system includes: obtaining a sequence of frames of data from an acquisition system; comparing successive frames of the data for portions that change between frames; determining whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data; defining a 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data; tracking a movement of the interaction medium to generate a plurality of 3D positions of the interaction medium; detecting movement of the interaction medium from inside to outside the 3D interaction zone at a boundary 3D position; shifting the 3D interaction zone relative to the boundary 3D position; computing a plurality of computed positions based on the 3D positions; and supplying the computed positions to control an application.

Systems and methods for remapping three-dimensional gestures onto a finite-size two-dimensional surface

A method for operating a real-time gesture based interactive system includes: obtaining a sequence of frames of data from an acquisition system; comparing successive frames of the data for portions that change between frames; determining whether any of the portions that changed are part of an interaction medium detected in the sequence of frames of data; defining a 3D interaction zone relative to an initial position of the part of the interaction medium detected in the sequence of frames of data; tracking a movement of the interaction medium to generate a plurality of 3D positions of the interaction medium; detecting movement of the interaction medium from inside to outside the 3D interaction zone at a boundary 3D position; shifting the 3D interaction zone relative to the boundary 3D position; computing a plurality of 2D positions based on the 3D positions; and supplying the 2D positions to control an application.

Systems and methods for initializing motion tracking of human hands within bounded regions are disclosed. One embodiment includes: a processor; reference and alternate view cameras; and memory containing a plurality of templates that are rotated and scaled versions of a base template. In addition, a hand tracking application configures the processor to: obtain reference and alternate view frames of video data; generate a depth map; identify at least one bounded region within the reference frame of video data containing pixels having distances from the reference camera that are within a specific range of distances; determine whether any of the pixels within the at least one bounded region are part of a human hand; track the motion of the part of the human hand in a sequence of frames of video data obtained from the reference camera; and confirm that the tracked motion corresponds to a predetermined initialization gesture.

Systems and methods for initializing motion tracking of human hands using template matching within bounded regions determined using a depth map

Systems and methods for tracking human hands using parts based template matching are described. One embodiment of the invention includes a processor, a reference camera and memory containing: a hand tracking application; and a finger template including an edge features template. In addition, the hand tracking application configures the processor to: detect at least one candidate finger in a frame of video data received from the reference camera, where each candidate finger is a grouping of pixels identified by searching the frame of video data for a grouping of pixels that have image gradient orientations that match the edge features of the finger template accounting for rotation and scaling differences; and verify the correct detection of a candidate finger by confirming that the colors of the pixels within the grouping of pixels identified as a candidate finger satisfy a skin color criterion.

Systems and methods for tracking human hands using parts based template matching

Systems and methods for initializing motion tracking of human hands are disclosed. One embodiment includes a processor; a reference camera; and memory containing: a hand tracking application; and a plurality of edge feature templates that are rotated and scaled versions of a base template. The hand tracking application configures the processor to: determine whether any pixels in a frame of video are part of a human hand, where a part of a human hand is identified by searching the frame of video data for a grouping of pixels that have image gradient orientations that match the edge features of one of the plurality of edge feature templates; track the motion of the part of the human hand visible in a sequence of frames of video; confirm that the tracked motion corresponds to an initialization gesture; and commence tracking the human hand as part of a gesture based interactive session.

Growing community of inventors

Sunnyvale, CA, United States of America

Michele Stoppa