Toyota Research Institute, Inc.

University of Michigan

John H Gideon

Simon A I Stent

Guy Rosman

Jie Li

Luke S Fletcher

Emily Mower Provost

Thomas Balch

Soheil Khorram

Melvin Mcinnis

Katherine Anne Matton

Zhijian Liu

Systems and methods for training remote photoplethysmography ('PPG') models that outputs a subject PPG signal based on a subject video clip of a subject are described herein. The system may have a processor and a memory in communication with the processor. The memory may include a training module having instructions that, when executed by the processor, cause the processor to train the remote PPG model in a self-supervised contrastive learning manner using an unlabeled video clip having a sequence of images of a face of a person.

System and method for training a remote photoplethysmography model

Systems and methods for training a model are described herein. In one example, a system for training the model includes a processor and a memory in communication with the processor having a training module. The training module has instructions that cause the processor to determine a contrastive loss using a self-supervised contrastive loss function, adjust, based on the contrastive loss, model weights a visual backbone that generated feature maps and/or a textual backbone that generated feature vectors. The training module also has instructions that cause the processor to determine a localized loss using a supervised loss function that compares an image-caption attention map with visual identifiers and adjust, based on the localized loss, the model weights the visual backbone and/or the textual backbone.

System and method for training a model using localized textual supervision

A method of predicting a mood state of a user may include recording an audio sample via a microphone of a mobile computing device of the user based on the occurrence of an event, extracting a set of acoustic features from the audio sample, generating one or more emotion values by analyzing the set of acoustic features using a trained machine learning model, and determining the mood state of the user, based on the one or more emotion values. In some embodiments, the audio sample may be ambient audio recorded periodically, and/or call data of the user recorded during clinical calls or personal calls.

Automatic speech-based longitudinal emotion and mood recognition for mental health treatment

A control system, computer-readable storage medium and method of preventing occlusion of and minimizing shadows on the driver's face for driver monitoring. The system includes a steering wheel, a plurality of fiberscopes arranged evenly spaced around the steering wheel, and one or more video cameras arranged at remote ends of the plurality of fiberscopes. Distal ends of the fiberscopes emerge to a surface of the steering wheel through holes that are perpendicular to an axis of rotation of the steering wheel. Each of the distal ends of the fiberscopes includes a lens. The system includes a plurality of light sources and an electronic control unit connected to the one or more video cameras and the light sources.

Multiple in-cabin cameras and lighting sources for driver monitoring

Systems and methods for monitoring a user by a plurality of image sensors are disclosed. The system includes a plurality of image sensors that output content descriptive of one or more visual attributes of the user, where each of the image sensors are positioned at a unique angle facing a user. The system also includes one or more processors and one or more non-transitory memory modules communicatively coupled to the one or more processors. The non-transitory memory modules storing machine-readable instructions that, when executed, cause the one or more processors to receive the content from the plurality of image sensors, and assign respective weights to each of the plurality of image sensors based on the content. The respective weights of the plurality of image sensors are actively re-weighted over a period of time. The processors are also caused to merge the content of the plurality of image sensors together.

Growing community of inventors

Howell, MI, United States of America

John H Gideon