Perceptive Pixel, Inc.

Meta Platforms Technologies, LLC

Facebook Technologies, LLC

Jason Reisman

Philip L Davidson

Jefferson Y Han

Adam L Berenzweig

Christopher Anderson

Daniel Z Wetmore

Robert Cochran

Austin Ha

Zaina Edelson

Nitzan Bartov

Naor Brown

Joshua Duyan

Patrick Kaifosh

Alexandre Barachant

Jeffrey Scott Seely

Paulo Jose Telo Coelho

Szeyin Lee

Luke O'Connor

Qiushi Mao

Robert John DiMaiolo

Ran Rubin

Edith Comas

Shaker Islam

Jasmine Stone

The various implementations described herein include methods and systems for using a multi-stage in-air hand gesture to activate user-interface interactions in a way that ensures low-false positive rates. In one aspect, a method includes, while a gating gesture is maintained, receiving a first indication of performance of an adjustment gesture of a first magnitude directed to a user interface object associated with a plurality of values. The method further includes, in response to receiving the first indication, adjusting the user interface object to have a first state after moving through some of the plurality of values based on the first magnitude. The method also includes, after receiving an indication of a release of the gating gesture, in response to receiving a second indication of performance of the adjustment gesture, forgoing adjusting the user interface object such that the user interface object continues to have the first state.

Multi-stage gestures detected based on neuromuscular-signal sensors of a wearable device to activate user-interface interactions with low-false positive rates, and systems and methods of use thereof

The various implementations described herein include methods and systems for using hand gestures detected at a wearable device to navigate a user interface. An example method includes receiving, via one or more neuromuscular-signal sensors of a wrist-wearable device worn by a user, data generated during performance of an in-air wrist movement by a wrist of the user. The method also includes moving a point of focus on the user interface in accordance with the in-air wrist movement; and receiving, via the one or more neuromuscular-signal sensors, additional data generated during performance of an in-air gesture by the user. The method further includes determining that the in-air gesture is an execution gesture; and executing a command corresponding to the execution gesture.

Navigating a user interface using in-air gestures detected via neuromuscular-signal sensors of a wearable device, and systems and methods of use thereof

The disclosed computer-implemented method may include presenting, via a user interface, a sensory cue, and receiving, from neuromuscular sensors of a wearable device, various neuromuscular signals generated by a user wearing the wearable device, where the user generates the neuromuscular signals in response to the sensory cue being presented to the user via the user interface. The method may also include interpreting the received neuromuscular signals as input commands with respect to the sensory cue provided by the user interface, such that the input commands initiate performance of specified tasks within the user interface. The method may also include performing the specified tasks within the user interface according to the interpreted input commands. Various other methods, systems, and computer-readable media are also disclosed.

Systems, methods, and interfaces for performing inputs based on neuromuscular control

Methods and apparatus for controlling a physical object in an environment based, at least in part, on neuromuscular signals. The method comprises recording a plurality of neuromuscular signals from a plurality of neuromuscular sensors arranged on one or more wearable devices worn by a user, receiving a selection of a physical object within the environment, and controlling, based at least in part on the plurality of neuromuscular signals and/or information based on the plurality of neuromuscular signals, an operation of the selected object within the environment.

Neuromuscular control of physical objects in an environment

Placement by one or more input mechanisms of a touch point on a multi-touch display device that is displaying a three-dimensional object is detected. A two-dimensional location of the touch point on the multi-touch display device is determined, and the touch point is matched with a three-dimensional contact point on a surface of the three-dimensional object that is projected for display onto the image plane of the camera at the two-dimensional location of the touch point. A change in applied pressure at the touch point is detected, and a target depth value for the contact point is determined based on the change in applied pressure. A solver is used to calculate a three-dimensional transformation of the three-dimensional object using an algorithm that reduces a difference between a depth value of the contact point after object transformation and the target depth value.

3D manipulation using applied pressure

Placement by one or more input mechanisms of a touch point of a mufti-touch display device that is displaying a three-dimensional object is detected. A two-dimensional location of the touch point on the multi-touch display device is determined, and the touch point is matched with a three-dimensional contact point on a surface of the three-dimensional object that is projected for display onto the image plane of the camera at the two-dimensional location of the touch point. A change in applied pressure at the touch point is detected, and a target depth value for the contact point is determined based on the changed in applied pressure. A solver is used to calculate a three-dimensional transformation of the three-dimensional object using an algorithm that reduces a difference between a depth value of the contact point after object transformation and the target depth value.

A three-dimensional object is initially transformed using an initial three-dimensional transformation in response to the detected movement of at least one touch point of a set of touch points placed on a multi-touch display device, each touch point in the set being matched with a contact point on the surface of the object. If the initially transformed object does not correspond to a rotational exhaustion situation, displaying the initially transformed object on the multi-touch display device. If the initially transformed object corresponds to a rotational exhaustion situation, calculating a corrected three-dimensional transformation of the object by using an algorithm that fixes at least one degree of freedom of object orientation to a value corresponding to an orientation of the object prior to the detected movement of the at least one touch point. Transforming the object using the corrected three-dimensional transformation and displaying the transformed object.

Addressing rotational exhaustion in 3D manipulation

A three-dimensional transformation of a three-dimensional object is calculated by a solver using an algorithm that reduces deviation between projected two-dimensional locations of three-dimensional contact points on a surface of the object after object transformation and two dimensional locations of touch points placed on a multi-touch display device. The algorithm is biased to generate a three-dimensional transformation that preferentially rotates the object in one direction over an opposite direction when all three-dimensional contact points are located at substantially the same depth along a Z-axis away from an image plane of the multi-touch display device and when transformation is then triggered by touch points moving such that a Euclidean distance between at least two of the touch points decreases.

Resolving ambiguous rotations in 3D manipulation

Manipulating a three-dimensional object displayed in a multi-touch display device includes displaying a three-dimensional object in two dimensions on the multi-touch display device. Placement of one or more touch points on the multi-touch display device is detected. For each detected touch point, a location of the touch point on the multi-touch display device is determined and a matching contact point on a surface of the three-dimensional object that is displayed by the multi-touch display device at the location of the touch point is also determined. Movement of at least one of the touch points is detected. Subsequent to the detected movement of the one or more touch points, a three-dimensional transformation of the three-dimensional object is determined that results in a display in which the contact points on the surface of the three-dimensional object remain displayed substantially at the locations of their matching touch points.

Screen-space formulation to facilitate manipulations of 2D and 3D structures through interactions relating to 2D manifestations of those structures

Techniques for constraining motion of 3D objects displayed on a 2D display interface are described. Touch points are placed by a user on the 2D display interface to manipulate a displayed object. Each touch point is matched with a contact point on the surface of the object. The motion of the object is restricted by adding penalty terms to an energy equation that includes terms that measure deviation between the screen-space location of the touch points and that of their matching contact points. The penalty terms measure deviation from an ideal value. In response to movement of at least one touch point to a new screen-space location, a transformation of the object is determined by applying an algorithm that operates on the energy equation to reduce deviations between the screen-space location of the touch points and that of their matching contact points while also reducing deviation from the ideal value or values set by the penalty term or terms.

Constraining motion in 2D and 3D manipulation

Each touch point placed on a multi-touch display device is matched with a contact point on a surface of a three-dimensional object displayed on the multi-touch display device. A three-dimensional transformation of the object is calculated by a solver using an algorithm that reduces deviation between projected two-dimensional locations of the three-dimensional contact points after object transformation and two dimensional locations of their matching touch points. The solver, in calculating the three-dimensional transformation, assigns a weighting value to each touch point to distribute an aggregate amount of deviation calculated by the solver among the touch points such that a magnitude of the deviation between the projected two-dimensional location of one of the contact points and that of its matching touch point is different from a magnitude of the deviation between the projected two dimensional location of another contact point and that of its matching touch point.

Growing community of inventors

Brooklyn, NY, United States of America

Jason Reisman