Microsoft Technology Licensing, LLC

Priyanka Vikram Sinha

Amit Srivastava

Derek Martin Johnson

Gencheng Wu

Konstantin Seleskerov

Ji Li

Siliang Kang

Huey-Ru Tsai

Huakai Liao

Kevin Dara Khieu

Brittany Elizabeth Mederos

Techniques performed by a data processing system include establishing an online presentation session for conducting an online presentation, receiving first media streams comprising presentation content from the first computing device, receiving second media streams from the second computing devices of a subset of the plurality of participants, the second media streams including audio content, video content, or both of the subset of the plurality of participants, analyzing the first media streams using first machine learning models to generate feedback results, analyzing the set of second media streams to identify first reactions by the participants to obtain reaction information, automatically analyzing the feedback results and the reactions to identify discrepancies between the feedback results and the reactions, and automatically updating one or more parameters of the machine learning models based on the discrepancies to improve the suggestions for improving the online presentation.

Techniques for presentation analysis based on audience feedback, reactions, and gestures

The present disclosure relates to processing operations configured to provide processing that automatically analyzes acoustic signals from attendees of a live presentation and automatically triggers corresponding reaction indications from results of analysis thereof. Exemplary reaction indications provide feedback for live presentations that can be presented in real-time (or near real-time) without requiring a user to manually take action to provide any feedback. As a non-limiting example, reaction indications may be presented in a form that is easy to visualize and understand such as emojis or icons. Another example of a reaction indication is a graphical user interface (GUI) notification that provides a predictive indication of user intent derived from analysis of acoustic signals. Further examples described herein extend to training and application of artificial intelligence (AI) processing, in real-time (or near real-time), that is configured to automatically analyze acoustic features of audio streams and automatically generate exemplary reaction indications.

Automatic reaction-triggering for live presentations

An intelligent speech assistant receives information collected while a user is speaking. The information can comprise speech data, vision data, or both, where the speech data is from the user speaking and the vision data is of the user while speaking. The assistant evaluates the speech data against a script which can contain information that the user should speak, information that the user should not speak, or both. The assistant collects instances where the user utters phrases that match the script or instances where the user utters phrases that do not match the script, depending on whether phases should or should not be spoken. The assistant evaluates vision data to identify gestures, facial expressions, and/or emotions of the user. Instances where the gestures, facial expressions, and/or emotions are not appropriate to the context are flagged. Real-time prompts and/or a summary is presented to the user as feedback.

Speaking technique improvement assistant

Growing community of inventors

Sunnyvale, CA, United States of America

Priyanka Vikram Sinha