Samsung Electronics Co., Ltd.

University of California

Jilong Kuang

Jun Gao

Md Mahbubur Rahman

Ebrahim Nematihosseinabadi

Viswam Nathan

Korosh Vatanparvar

Tousif Ahmed

Sean Bornheimer

Li Yan Zhu

Nathan Robert Folkman

Anh Minh Dinh

Daniel Waddington

Nasson Boroumand

Chen Tian

David W Mortara

Juan Colmenares

Mahbubur Rahman

Mohsin Ahmed

Jeong Woo Kim

Jeffrey E Olgin

Ebrahim Nemati

Migyeong Gwak

Daniyal Liaqat

Shibo Zhang

Wenchuan Wei

Bashima Islam

Jun (Alex) Gao

Nazir Saleheen

Md Juber Rahman

Soujanya Chatterjee

Pulmonary assessment based on speech can include identifying one or more audio features and speech patterns of a user's speech. A cognitive burden associated with the user's speech can be determined. A pulmonary condition of the user can be determined based on predetermined correlations between the one or more audio features and speech patterns of the user's speech, the cognitive burden, and a respiratory airway condition.

Speech-based pulmonary assessment

Detecting and identifying a predetermined health event can include detecting a potential occurrence of the predetermined health event for a user by processing in real-time motion signals corresponding to motion of the user. A likelihood that the potential occurrence is an actual occurrence of the predetermined health event can be determined based on template matching of the motion signals. In response to determining that the likelihood exceeds a predetermined threshold, audio signals coinciding in time with the motion of the user can be processed using one or more layers of a multilayered audio event classifier.

Multilayered determination of health events using resource-constrained platforms

A multimodal, contactless vital sign monitoring system is configured to perform the following operations. Images are received from a video capture device. An image of a subject is identified within the images. The image of the subject is segmented into a plurality of segments. A first analysis is performed on the plurality of segments to identify a color feature. A second analysis is performed of the plurality of segments to identify a motion feature. Using a combination of the color feature and the motion feature a plurality of vital signs for the subject are determined. The first analyzing and the second analyzing are performed in parallel. The plurality of vital signs include one or more of heart rate, respiration rate, oxygen saturation, heart rate variability, and atrial fibrillation.

Multimodal contactless vital sign monitoring

Detecting an adverse health event and responding to an emergency event can include updating a current context profile of a user based on signals generated by one or more sensors of a device, the signals generated in response to one or more physically measurable phenomena associated with the user. A context-aware baseline profile having a same context as the current context profile can be selected, the context-aware baseline profile selected from a plurality of context-aware baseline profiles having different contexts. Features of the current context profile can be compared with corresponding features of the context-aware baseline profile of the user having the same context as the current context profile. A remedial action can be initiated in response to recognizing, based on the comparing, an occurrence of a possible adverse health event affecting the user.

Timely detection and response to context-specific health events

Passively monitoring a user's breathing with a device can include identifying breathing modes of the user's breathing and responsive to detecting a trigger mode based on the identifying, generating an instruction adapted to the trigger mode. The instruction can be conveyed to the user via the device. The monitoring can include determining phases of the user's breathing with the device. Determining the phases can include receiving acoustic signals generated by an acoustic sensor in response to a user's breathing and generating acoustic data comprising features extracted from the acoustic signals. Phases of the user's breathing can be determined by classifying the acoustic data using a machine learning model trained based on signal processing of motion signals generated by a motion sensor in response to human breathing motions. Though trained using signal processing of motion signals, the machine learning model is trained to classify acoustic data.

Breathing measurement and management using an electronic device

Personalized vertigo rehabilitation and treatment can include generating a sequence of sounds that are modulated to perceptually emanate from one or more distinct locations in three-dimensional space relative to a user. The sequence can be predetermined to guide head movements of the user in a prescribed manner to mitigate vertigo experienced by the user. The user's head movements can be tracked as the user responds to the sequence of sounds. A signal can be conveyed to the user in response to detecting a predetermined head movement by the user.

Personalized vertigo rehabilitation

A method includes receiving, by an electronic device, sensor data during a spirometry test of a user, the sensor data comprising audio data of the user, image data of a face of the user, and distance data of a distance from the face of the user. The method also includes obtaining, by the electronic device, at least one measured parameter associated with the spirometry test that is determined using at least one of the audio data, the image data, or the distance data, wherein the at least one measured parameter is correlated with an amount of air volume exchange or an amount of exhalation force by the user during the spirometry test. The method further includes providing, by the electronic device, real time feedback during the spirometry test, the real time feedback based on the at least one measured parameter, the real time feedback indicating whether or not the user is performing the spirometry test correctly.

System and method for conducting on-device spirometry test

Pulmonary function estimation can include detecting one or more cough events from a time series of audio signals generated by an electronic device of a user. Based on the one or more cough events, one or more lung function metrics of the user can be determined.

Pulmonary function estimation

In particular embodiments, a method includes accessing, by a computing device, sensor data from a sensor of a client device corresponding to a user. The method further includes determining one or more segments of sensor data and selecting at least one segment of sensor data. The method further concludes determining, based at least on the selected segments of sensor data, one or more features corresponding to the user's pulmonary condition. The method further includes determining, based at least on the determined features, an assessment of the user's pulmonary condition.

Methods and systems for pulmonary condition assessment

A method includes estimating, by a consumer electronic device, an atrial fibrillation (AF) burden of a subject based on multiple measurements passively collected by at least one sensor associated with the consumer electronic device while the subject is in a free-living environment. The method also includes outputting, by the consumer electronic device, an AF burden notification associated with the subject based on the estimated AF burden. The method further includes outputting, by the consumer electronic device, an AF burden recommendation based on the estimated AF burden.

Systems and methods for atrial fibrillation burden estimation, notification and management in daily free-living scenarios

A method includes receiving, by an electronic device, sensor data during a spirometry test of a user, the sensor data comprising audio data of the user and distance data of a distance from a face of the user to the electronic device. The method also includes obtaining, by the electronic device, an amount of air volume exchange and at least one pulmonary health parameter that are determined based on the audio data and the distance data. The method also includes presenting an indicator on a display of the electronic device for use by the user or a medical provider, the indicator representing the amount of air volume exchange.

A method includes obtaining, by an electronic device, an audio segment comprising one or more audio events of a target subject. The method also includes extracting, by the electronic device, audio embeddings from the one or more audio events using an embedding model, the embedding model comprising a trained machine learning model. The method further includes comparing, by the electronic device, the extracted audio embeddings with a match profile of the target subject, the match profile generated during an enrollment stage. The method also includes generating, by the electronic device, a label for the audio segment based on whether or not the extracted audio embeddings match the match profile, wherein the label enables correlation of the audio segment with the target subject for monitoring a health condition of the target subject.

System and method for passive subject specific monitoring

Adaptive respiratory condition assessment can include capturing signals generated by sensors of a portable device positioned to sense a user's body. The sensors can generate the signals in response to sensor-detected movements of the user's body measured along multiple axes and corresponding to lung activity in the user's body during respiratory cycles. A preferred axis of measurement can be selected using signal processing performed by a signal processor embedded in the portable device. Waveforms generated from signals corresponding to signals generated in response to movements of the user's body measured along the preferred axis can be filtered for extracting biomarkers from the waveforms using the signal processor. The one or more biomarkers extracted correspond to the lung activity. Based on the biomarkers, a respiratory condition of the user can be determined. A notification based on the respiratory condition as determined can be generated and conveyed with the portable device.

Adaptive respiratory condition assessment

Continuously monitoring heart rhythm can include grouping, using computer hardware, a plurality of inter-beat intervals (IBI) data for a user into a plurality of epochs, wherein each epoch includes a subset of the IBI data corresponding to a predetermined time span. For each epoch, a selected feature set selected from a plurality of feature sets is extracted based on a determination of temporal consistency of the epoch. A plurality of epoch classifications may be generated for the epochs using a selected feature processor, wherein each epoch classification indicates whether arrhythmia is detected for the epoch from which the epoch classification is generated. The selected feature processor is selected from a plurality of different feature processors on a per-epoch basis based on the selected feature set extracted from the epoch. An indication of arrhythmia may be output, via an output device of the computer hardware, based on the epoch classifications.

Non-invasive continuous heart rhythm monitoring based on wearable sensors

A method for pulmonary condition monitoring includes selecting a phrase from an utterance of a user of an electronic device, wherein the phrase matches an entry of multiple phrases. At least one speech feature that is associated with one or more pulmonary conditions within the phrase is identified. A pulmonary condition is determined based on analysis of the at least one speech feature.

Growing community of inventors

San Jose, CA, United States of America

Jilong Kuang