Snap Inc.

Meir Meshulam

Assif Ziv

Marcos Jimenez

Mark Kliger

Elizabeth Izakson

Methods and systems are disclosed for iteratively training a user and a ML model to produce accurate inner speech outputs. The methods and systems access a ML model and perform a first training iteration in which EMG data corresponding to inner speech is processed by the machine learning model to decode the EMG data into a set of predicted phonemes, phoneme sounds, words or phrases. The methods and systems present the set of predicted phonemes, phoneme sounds, words or phrases to the user and form a first set of training data comprising the set of predicted phonemes, phoneme sounds, words or phrases, the EMG data, and the set of specified phonemes, phoneme sounds, words or phrases as ground truth information. The methods and systems update parameters of the ML model based on the first set of training data prior to starting a second training iteration.

Inner speech iterative learning loop

Methods and systems are disclosed for training users to produce inner speech. The system presents a prompt on a display device with one or more instructions for a user to produce inner speech and, in response to presenting the prompt, monitors EMG data to detect existence of inner speech. The system, in response to detecting existence of the inner speech, presents on the display device an indication that inner speech has been detected in the EMG data.

Training users to produce inner speech

Methods and systems are disclosed for training a machine learning (ML) model to detect inner speech. The system collects, by an electromyograph (EMG) communication device used by a user, a first set of EMG signals over a first time interval. The system generates a first plurality of features based on the first set of EMG signals and generates a first probability associated with presence of inner speech by processing the first plurality of features with a machine learning (ML) model. The system compares the first probability generated by the ML model to a specified threshold and detects presence of the inner speech of the user in response to determining that the first probability generated by the ML model transgresses the specified threshold.

Growing community of inventors

Princeton, NJ, United States of America

Meir Meshulam