Sony Depthsensing Solutions Sa/nv

Softkinetic Software

Laurent Guigues

Julien Thollot

Gilles Pinault

Utku Salihoglu

Aliaksandr Kamovich

Xavier Baele

Kevin Simons

Javier Martinez Gonzalez

Jeremie Roy

Antoine Rennuit

Michel Samuel

Olivier Feys

Carlos Souza

Yunshu Hou

Carlos Augusto Pinheiro De Sousa

Described herein is a method for enabling human-to-computer three-dimensional hand gesture-based natural interactions from depth images provided by a range finding imaging system. The method enables recognition of simultaneous gestures from detection, tracking and analysis of singular points of interests on a single hand of a user and provides contextual feedback information to the user. The singular points of interest of the hand: include hand tip(s), fingertip(s), palm center and center of mass of the hand, and are used for defining at least one representation of a pointer. The point(s) of interest is/are tracked over time and are analyzed to enable the determination of sequential and/or simultaneous 'pointing' and 'activation' gestures performed by a single hand.

Method and system for human-to-computer gesture based simultaneous interactions using singular points of interest on a hand

Described herein is a method for detecting keypoints in three-dimensional images in which a three-dimensional image of a scene captured by a depth sensing imaging system is processed using a distance-independent keypoint filter. Keypoints are derived from the three-dimensional image by determining a mean shift field and using x- and y-components of the mean shift field to derive intersections of 0-isolines thereof. Positive and negative keypoints or nodes are connected to one another, positive to positive and negative to negative, to form a keygraph structure.

Distance-independent keypoint detection

Described herein is a method for image processing in which a captured image is processed to provide a background image which is updated in accordance with relative movement between foreground and background in successive captured images.

Method for foreground and background determination in an image

Described herein is method for applying noise reduction to multi-dimensional input data by applying a multi-dimensional approximation function to each point in the input data whilst maintaining definition for each discontinuity () present in the input data. A control lattice including a plurality of control points () is superposed on the input data such that, for each data point, a number of control points surround the data point (). For each control point (), a value which is a function of neighbouring data points is computed, and from, at least values of the neighbouring control points, determining an approximation function, which when applied to the input data provides filtered output data. Weighting is applied to at least control point associated with a data point in the vicinity of a discontinuity to maintain the sharpness of the discontinuity during the noise reduction process.

Or relating to filters

Described herein is a method and a system for providing efficient and complementary natural multi-modal gesture based interaction with a computerized system which displays visual feedback information on a graphical user interface on an interaction surface. The interaction surface is within the frustum of an imaging device comprising a single sensing system. The system uses the single sensing system for detecting both touch gesture interactions with the interaction surface () and three-dimensional touch-less gesture interactions in areas or volumes above the interaction surface performed by hands of a user. Both types of interaction are associated contextually with an interaction command controlling the computerized system when the gesture has been detected. The system comprises preferably a projection system for displaying the graphical user interface and visual feedback on the interaction surface, the projection system being locatable on the same side or on the opposite side of the interaction surface to the sensing system.

Multi-modal gesture based interactive system and method using one single sensing system

Described herein is a method for signal processing in which noise in an input signal comprising an intensity which is a function of at least a first coordinate and a second coordinate. A noise reduction or de-noising process is applied to the input signal in respect of the first coordinate to generate an intermediate de-noised signal. A second noise reduction or de-noising process is applied to the intermediate de-noised signal in respect of the second coordinate to generate an output de-noised signal. For each coordinate, noise reduction processes are applied in two directions, the results of these processes being averaged to provide the de-noised signals. Weighting is applied in accordance with the detection of an edge within the signal used as input in the de-noising process.

Method for signal processing

Described herein is a method for correcting defective depth values in depth map images. Defective values correspond to 'noisy pixels' located on continuous flat surfaces and interpolated 'flying pixels' located along an edge between a foreground object and a background object. The method comprising the steps of accessing a depth map of a scene which includes the foreground and background objects, detecting and identifying noisy and interpolated depth measurements within the depth map using a method, defining and applying a correction to each of the detected defective noisy and interpolated depth measurements using a specifically defined weighted correction factor. By providing the corrected defective depth values in depth map images, edges are sharpened in depth and continuous surfaces are flattened, enabling higher efficiency and robustness of further image processing.

Depth measurement quality enhancement

Described herein is a method and system for marker-less three-dimensional modelling, fitting and tracking of a skeletal representation of an object in a three-dimensional point cloud. In particular, it concerns the tracking of a human user skeletal representation with respect to time. The method comprises inputting a three-dimensional point cloud derived from a depth map; predetermining a set of control points representing the skeleton of the user, determining a start-up skeleton pose, obtaining an orthographic representation of the user 3D point cloud projected onto a grid by sampling the 3D point cloud with a predetermined static size, determining a set of curvature centers points approximating central axes of main parts of the user, determining the torso plane, and refining and/or defining the principal direction of the body. The method comprises then the step of performing iterative local and global fittings of the set of control points onto the user 3D point cloud and the associated data such as the curvature center points, using topological and geometric constraints so that to track skeleton posture along the time. Stabilizing the skeleton pose; resolving ambiguities; and providing a suitable output are then the last steps of a preferred embodiment of the invention.

Three-dimensional object modeling fitting and tracking

Described herein is a method for detecting, identifying and tracking hand, hand parts, and fingers on the hand () of a user within depth images of a three-dimensional scene. Arms of a user are detected, identified, segmented from the background of the depth images, and tracked with respect to time. Hands of the user are identified and tracked and the location and orientation of its parts, including the palm and the fingers () are determined and tracked in order to produce output information enabling gesture interactions.

Three dimensional close interactions

The present invention relates to a method for tracking at least one object in a sequence of frames, each frame comprising a pixel array, wherein a depth value is associated to each pixel. The method comprises grouping at least some of said pixels of each frame into several regions, grouping said regions into clusters (B, . . . , B) of interconnected regions; and determining that a cluster (B, . . . , B) which is adjacent to another cluster (B) in a two-dimensional projection belongs to an object partially occluded by said other cluster (B) if it has a different depth value than said other cluster (B).

Growing community of inventors

Brussels, Belgium

Laurent Guigues