Dell Products Inc.

Jonathan Whitson

Michael David Shepherd

Joseph Michael Whitson

Satheeshkumar Ramasamy

Joseph Tansey

Techniques are provided for automatically generating language model prompts using predicted regions of interest. One method comprises obtaining a focus of a user interacting with an image and/or a virtual environment; determining a predicted region of interest of the user based on the focus, wherein the predicted region of interest has associated metadata indicating an item of interest, in the image and/or the virtual environment, to the user; providing a query to an information retrieval system that generates query results, wherein the query is based on the item of interest to the user obtained from the metadata; determining a prompt for a language model based on the query results; and providing a message, automatically generated by the language model based on the prompt, to the user. The predicted region of interest of the user may be obtained by tracking a movement of an eye of the user.

Automatically generating language model prompts using predicted regions of interest

Techniques are provided for orienting a digital human towards an isolated human speaker. One method comprises obtaining coordinates associated with a human speaker; storing the coordinates as a variable in a memory of a multi-dimensional graphics engine; instantiating a multi-dimensional plane within the multi-dimensional graphics engine; positioning a multi-dimensional shape in the multi-dimensional plane based on the stored coordinates; and binding a digital human to the multi-dimensional shape to orient the digital human towards a position of the multi-dimensional shape. The coordinates may be obtained by comparing text obtained from one or more video streams to text obtained from one or more audio signals.

Orienting digital humans towards isolated speaker

A system can determine, from a video of an online session, respective bounding boxes of text names of people, wherein the text names are presented in the video, and wherein images of the people are present in the video. The system can determine, from the video, respective faces of the people. The system can associate a first bounding box of the bounding boxes with a first face of the faces based on the first bounding box satisfying a function of distance with respect to the first face among the faces. The system can extract a name from the first bounding box via optical character recognition. The system can extract a subportion of the video that comprises the first face. The system can store an association between the name and the subportion of the video that comprises the first face.

Extracting facial imagery from online sessions

A system can identify time-series bio telemetry data that corresponds to a first user of an online session, wherein the time-series bio telemetry data identifies a first user engagement of the first user. The system can identify a first video recording of the online session, wherein the first video recording is representative of a first video of the first user. The system can input the time-series bio telemetry data and the first video recording as a first input to a neural network model to produce a trained neural network model, wherein the trained neural network model is configured to classify a second user engagement from a second video recording representative of a second video of a second user. The system can classify the second user engagement of the second user comprising inputting the second video recording as a second input to the trained neural network model.

Information extraction from live online sessions

A system can, in response to determining to capture bio telemetry data associated with client devices, synchronize respective second clock times of respective client devices with a first clock time maintained by a network time protocol server, wherein the respective client devices are configured to capture the bio telemetry data of respective users associated with the client devices, wherein the respective users are associated with respective user accounts. The system can synchronize the bio telemetry data of the respective users based the respective second clock times.

Bio-telemetry extraction from online sessions

A system can receive respective bio telemetry data from a group of people in an online session, wherein the bio telemetry data comprises alpha wave data and beta wave data. The system can determine a current moving average for the current time period based on a numerical combination of respective beta wave data of the bio telemetry data and respective alpha wave data of the bio telemetry data, wherein the numerical combination is scaled by a first weighting value, wherein the current moving average is decreased by a previous moving average for a previous time period, and wherein the previous moving average is scaled by a second weighting value. The system can, in response to determining that attentiveness by the group of people has decreased based on the previous moving average being greater than the current moving average, sending attentiveness notification data directed to an instructor user account.

Growing community of inventors

Oklahoma City, OK, United States of America

Jonathan Whitson