Facebook Technologies, LLC

Brett Jurman

Patrick Kaifosh

Adam L Berenzweig

Daniel Z Wetmore

Thomas Reardon

Christopher Osborn

Tudor Giurgica-Tiron

Computerized systems, methods, and computer-readable storage media storing code for implementing the methods are described for providing dynamically-updated musculoskeletal information. One such system includes a processor is programmed to: provide, as an input to a trained inference model, information based on a plurality of neuromuscular signals from a user and information based on at least one image of the user; determine, based on an output of the trained inference model, position information describing a spatial relationship between two or more connected musculoskeletal segments of the user and/or force information describing a force exerted by at least one musculoskeletal segment of the user; and output the position information and/or the force information.

Camera-guided interpretation of neuromuscular signals

Computerized systems, methods, and computer-readable storage media storing code for implementing the methods are provided for training an inference model used to generate a musculoskeletal representation. One such system includes a processor programmed to: determine, based on information obtained from at least one image, position information describing a spatial relationship between two or more connected musculoskeletal segments of a user; determine, based on a plurality of neuromuscular signals, force information; associate the position information with the force information; train an inference model to output a musculoskeletal representation consistent with the position information and/or the force information when neuromuscular input signals provided to the inference model have at least one predetermined characteristic, to produce an updated inference model; and cause the updated inference model to be stored in a memory.

Methods and systems for dynamically reconstructing handstate information based on multiple inputs are described. The methods and systems use data from multiple inputs including a plurality of neuromuscular sensors arranged on one or more wearable devices and one or more cameras. The multimodal data is provided as input to a trained statistical model. The methods and systems determine, based on the data from the multiple inputs, an estimate and representation of the spatial relationship between two or more connected segments of the musculoskeletal representation and force information describing a force exerted by at least one segment of the musculoskeletal representation. The methods and systems further update the computerized musculoskeletal representation based, at least in part, on the position information and the force information.

Growing community of inventors

New York, NY, United States of America

Brett Jurman