Intel Corporation

Shahar Fleishman

Mark Kliger

Gershom Kutliroff

Yaron Yanai

Alon Lerner

Amit Bleiweiss

Jonathan Epstein

Yotam Livny

Maoz Madmony

Jackie Assa

Naomi Ken Korem

Rana Hanocka

A system and method for close range object tracking are described. Close range depth images of a user's hands and fingers or other objects are acquired using a depth sensor. Using depth image data obtained from the depth sensor, movements of the user's hands and fingers or other objects are identified and tracked, thus permitting the user to interact with an object displayed on a screen, by using the positions and movements of his hands and fingers or other objects.

System and method for close-range movement tracking

A system, article, and method of recurrent semantic segmentation for image processing by factoring historical semantic segmentation.

Method and system of recurrent semantic segmentation for image processing

Hand dimensions are determined for hand and gesture recognition with a computing interface. An input sequence of frames is received from a camera. Frames of the sequence are identified in which a hand is recognized. Points are identified in the identified frames corresponding to features of the recognized hand. A value is determined for each of a set of different feature lengths of the recognized hand using the identified points for each identified frame. Each different feature length value is collected for the identified frames independently of each other feature length value. Each different feature length value is analyzed to determine an estimate of each different feature length, and the estimated feature lengths are applied to a hand tracking system, the hand tracking system for applying commands to a computer system.

Determination of hand dimensions for hand and gesture recognition with a computing interface

Convolutional Neural Networks are applied to object meshes to allow three-dimensional objects to be analyzed. In one example, a method includes performing convolutions on a mesh, wherein the mesh represents a three-dimensional object of an image, the mesh having a plurality of vertices and a plurality of edges between the vertices, performing pooling on the convolutions of an edge of a mesh, and applying fully connected and loss layers to the pooled convolutions to provide metadata about the three-dimensional object.

Application of convolutional neural networks to object meshes

Techniques are provided for context-based 3D scene reconstruction employing fusion of multiple instances of an object within the scene. A methodology implementing the techniques according to an embodiment includes receiving 3D image frames of the scene, each frame associated with a pose of a depth camera, and creating a 3D reconstruction of the scene based on depth pixels that are projected and accumulated into a global coordinate system. The method may also include detecting objects, based on the 3D reconstruction, the camera pose and the image frames. The method may further include classifying the detected objects into one or more object classes; grouping two or more instances of objects in one of the object classes based on a measure of similarity of features between the object instances; and combining point clouds associated with each of the grouped object instances to generate a fused object.

Three dimensional scene reconstruction based on contextual analysis

Techniques are provided for global (non-rigid) scan point registration between a scanned object and an associated model, from an arbitrary initial starting position, based on a combination of iterative coarse registration and fine registration. A methodology implementing the techniques according to an embodiment includes generating a model transformation based on a coarse registration between the model and the point scan. The method further includes calculating an alignment metric based on a distance measurement between the point scan and the transformed model. If the alignment metric exceeds a selected threshold value, a fine registration is performed between the transformed model and the point scan. Otherwise, the method continues by performing a random rotation of the model; a translation of the rotated model towards a centroid of the point scan; and iterating the coarse registration using the translated model until the alignment metric is achieved, after which the fine registration is performed.

Automatic global non-rigid scan point registration

Systems, apparatuses and/or methods may provide for generating a packing order of items within a container that consolidates the items into a reduced space. Items may be scanned with a three-dimensional (3D) imager, and models may be generated of the items based on the data from the 3D imager. The items may be located within minimal-volume enclosing bounding boxes, which may be analyzed to determine whether they may be merged together in one of their bounding boxes, or into a new bounding box that is spatially advantageous in terms of packing. If a combination of items is realizable and is determined to take up less space in a bounding box than the bounding boxes of the items considered separately, then they may be merged into a single bounding box. Thus, a spatially efficient packing sequence for a plurality of real objects may be generated to maximize packing efficiency.

Efficient packing of objects

Techniques are provided for 3D analysis of a scene including detection, segmentation and registration of objects within the scene. The analysis results may be used to implement augmented reality operations including removal and insertion of objects and the generation of blueprints. An example method may include receiving 3D image frames of the scene, each frame associated with a pose of a depth camera, and creating a 3D reconstruction of the scene based on depth pixels that are projected and accumulated into a global coordinate system. The method may also include detecting objects, and associated locations within the scene, based on the 3D reconstruction, the camera pose and the image frames. The method may further include segmenting the detected objects into points of the 3D reconstruction corresponding to contours of the object and registering the segmented objects to 3D models of the objects to determine their alignment.

3-dimensional scene analysis for augmented reality operations

Systems and methods for combining three-dimensional tracking of a user's movements with a three-dimensional user interface display is described. A tracking module processes depth data of a user performing movements, for example, movements of the user's hands and fingers. The tracked movements are used to animate a representation of the hand and fingers, and the animated representation is displayed to the user using three-dimensional display. Also displayed are one or more virtual objects with which the user can interact. In some embodiments, the interaction of the user with the virtual objects controls an electronic device.

System and method for combining three-dimensional tracking with a three-dimensional display for a user interface

Techniques related to pose estimation for an articulated body are discussed. Such techniques may include extracting, segmenting, classifying, and labeling blobs, generating initial kinematic parameters that provide spatial relationships of elements of a kinematic model representing an articulated body, and refining the kinematic parameters to provide a pose estimation for the articulated body.

Detection, tracking, and pose estimation of an articulated body

Techniques related to non-rigid transformations for articulated bodies are discussed. Such techniques may include repeatedly selecting target positions for matching a kinematic model of an articulated body, generating virtual end-effectors for the kinematic model and corresponding to the target positions, generating an inverse kinematics problem including a Jacobian matrix, and determining a change in kinematic model parameters based on the inverse kinematics problem until a convergence is attained.

Iterative closest point technique based on a solution of inverse kinematics problem

Techniques related to labeling component parts and detecting component properties in imaging data are discussed. Such techniques may include generating a feature vector including invariant features associated with an area of interest within an image of an object such as an image of a hand and providing a component label such as a hand part label for the area of interest based on an application of a machine learning classifier to the feature vector.

Growing community of inventors

Hod Hasharon, Israel

Shahar Fleishman