Xerox Corporation

Survi Kyal

Lalit Keshav Mestha

Beilei Xu

Himanshu Madhu

Sohail A Dianat

Barry Paul Mandel

Peter Johan Nystrom

Martin Edward Hoover

Graham Stephen Pennington

Jean-Philippe Yves Couderc

Gill Rafael Tsouri

What is disclosed is a system and method for selecting a region of interest for extracting physiological parameters from a video of a subject. In one embodiment the present method involves performing the following. First, time-series signals are received which have been generated by having processing image frames of a video of a subject captured using a single band video camera with a bandpass filter with a pass band in a wavelength range of 495-565 nm and/or 800-1000 nm. The regions of interest are areas where a plethysmographic signal can be detected by the camera. Each time-series signal is associated with a different region of interest. A signal strength is then calculated for each of the time-series signals. The region associated with the time-series signal having a highest signal strength is selected. The time-series signal associated with the selected region can be processed to extract a videoplethysmographic (VPG) signal containing physiological parameters.

Selecting a region of interest for extracting physiological parameters from a video of a subject

What is disclosed is a system and method for determining respiration rate from a video of a subject. In one embodiment, a video is received comprising plurality of time-sequential image frames of a region of a subject's body. Features of pixels are extracted from that region from each image frame and vectors formed from these features. Each image frame has an associated feature vector. A N&#xd7;M video matrix of the vectors of length N is constructed such that a total number of columns M in the video matrix correspond to a time duration over which the subject's respiration rate is to be determined. The video matrix is processed to obtain a matrix of eigenvectors where principal axes of variations due to motion associated with respiration are contained in a first few eigenvectors. One eigenvector is selected from the first few eigenvectors. A respiration rate is obtained from the selected eigenvector.

System and method for determining respiration rate from a video

What is disclosed is a system and method for extracting photoplethysmographic (PPG) signal (i.e., a cardiac signal) on a continuous basis from a time-series signals obtained from video images captured of a subject being monitored for cardiac function in a non-contact remote sensing environment involves the following. First, a time-series signal obtained from video images captured of a region of exposed skin where a photoplethysmographic (PPG) signal of a subject of interest can be registered. A sliding window is then used to define consecutive sequential segments of the time-series signal for processing. Each of the consecutive time-series signal segments is detrended such that low frequency variations and non-stationary components are removed. The detrended signals are processed to obtain, for each segment, a PPG signal. The PPG signal segments are then stitched together using a stitching method, as disclosed herein, to obtain a continuous PPG signal for the subject.

Continuous cardiac signal generation from a video of a subject being monitored for cardiac function

What is disclosed is a video system and method that accounts for differences in imaging characteristics of differing video systems used to acquire video of respective regions of interest of a subject being monitored for a desired physiological function. In one embodiment, video is captured using N video imaging devices, where N&#x2267;2, of respective regions of interest of a subject being monitored for a desired physiological function (i.e., a respiratory or cardiac function). Each video imaging device is different but has complimentary imaging characteristics. A reliability factor f is determined for each of the devices in a manner more fully disclosed herein. A time-series signal is generated from each of the videos. Each time-series signal is weighted by each respective reliability factor and combined to obtain a composite signal. A physiological signal can be then extracted from the composite signal. The processed physiological signal corresponds to the desired physiological function.

Video acquisition system for monitoring a subject for a desired physiological function

What is disclosed is a system and method for compensating for motion induced artifacts in physiological signals extracted from a video of a subject being monitored for a physiological function in a non-contact, remote sensing environment. The present method identifies a center frequency from a physiological signal obtained from processing a prior video segment. Since a moment to moment change in pulse frequency from one video segment to a next is not very large, signals obtained from sequential video segments can be repeatedly processed and an adaptive band-pass filter repeatedly re-configured and used to filter a next video segment, and so on. Using the teachings disclosed herein, a motion-compensated continuous cardiac signal can be obtained for the subject for continuous monitoring of the subject's cardiac function via video imaging. The teachings hereof provide an effective means for compensating for movement by the subject during video acquisition. Various embodiments are disclosed.

Using an adaptive band-pass filter to compensate for motion induced artifacts in a physiological signal extracted from video

What is disclosed is a system and method for compensating for motion during processing of a video of a subject being monitored for physiological function assessment. In one embodiment, image frames are received. Successive batches of N video frames are processed to isolate pixels associated with a body region of the subject where a physiological signal is registered by the camera. The pixels are processed to obtain a time-series signal for each batch. A determination is made whether movement during video acquisition of this batch of image frames exceeds a threshold level. If so then a size N of the next batch of image frames is changed to: N=N+M, where N+M&#x2266;N. Otherwise, a size N of a next batch is changed to: N=N&#x2212;M, where N&#x2212;M&#x2267;N. Thereafter, processing repeats in a real-time continuous manner as the next batch of the N image frames is received. Various embodiments are disclosed.

Compensating for motion during real-time batch processing of video for physiological function assessment

What is disclosed is a system for compensating for motion induced artifacts in a physiological signal obtained from multiple videos of a first and second region of interest a subject being monitored for a desired physiological function. At least one of the videos being of the first region and at least one of the videos being of the second region. The first region being at least one area of exposed skin where a desired signal corresponding to the physiological function can be registered by a video imaging device. The second region being an area where a movement by the subject is likely to induce motion artifacts into the signal. The videos are processed to isolate pixels associated with the first and second regions. Processed pixels of the isolated first regions to obtain a composite time-series signal. From the composite signal, a physiological signal corresponding to the physiological function is extracted.

Compensating for motion induced artifacts in a physiological signal extracted from multiple videos

What is disclosed is a system and method for compensating for motion induce artifacts in a physiological signal obtained from a video. In one embodiment, a video of a first and second region of interest of a subject being monitored for a desired physiological function is captured by a video device. The first region is an area of exposed skin wherein a desired signal corresponding to the physiological function can be registered. The second region is an area where movement is likely to induce motion artifacts into that signal. The video is processed to isolate pixels in the image frames associated with these regions. Pixels of the first region are processed to obtain a time-series signal. A physiological signal is extracted from the time-series signal. Pixels of the second region are analyzed to identify motion. The physiological signal is processed to compensate for the identified motion.

Compensating for motion induced artifacts in a physiological signal extracted from a single video

What is disclosed is a method for monitoring a subject for cardiac arrhythmia such as atrial fibrillation using an apparatus that can be comfortably worn by the subject around an area of exposed skin where a photoplethysmographic (PPG) signal can be registered. In one embodiment, the apparatus is a reflective or transmissive wrist-worn device with emitter/detector pairs fixed to an inner side of a band with at least one illuminator emitting source light at a specified wavelength band. The illuminator is paired to a respective photodetector comprising one or more sensors that are sensitive to a wavelength band of its paired illuminator. The photodetector measures intensity of sensed light emitted by a respective illuminator. The signal obtained by the sensors comprises a continuous PPG signal. The continuous PPG signal analyzed for peak-to-peak pulse points from which the existence of cardiac arrhythmia such as atrial fibrillation event can be determined.

Method and apparatus for monitoring a subject for atrial fibrillation

What is disclosed is a system and method for determining a subject of interest's arterial pulse transit time from time-varying source signals generated from video images. In one embodiment, a video imaging system is used to capture a time-varying source signal of a proximal and distal region of a subject of interest. The image frames are processed to isolate localized areas of a proximal and distal region of exposed skin of the subject. A time-series signal for each of the proximal and distal regions is extracted from the source video images. A phase angle is computed with respect to frequency for each of the time-series signals to produce respective phase v/s frequency curves for each region. Slopes within a selected cardiac frequency range are extracted from each of the phase curves and a difference is computed between the two slopes to obtain an arterial pulse transit time for the subject.

System and method for determining video-based pulse transit time with time-series signals

What is disclosed is a system and method for estimating cardiac pulse rate from a video of a subject being monitored for cardiac function. In one embodiment, batches of overlapping image frames are continuously received and processed by isolating regions of exposed skin. Pixels of the isolated regions are processed to obtain a time-series signal per region and a physiological signal is extracted from each region's time-series signals. The physiological signal is processed to obtain a cardiac pulse rate for each region. The cardiac pulse rate for each region is compared to a last good cardiac pulse rate from a previous batch to obtain a difference. If the difference exceeds a threshold, the cardiac pulse rate is discarded. Otherwise, it is retained. Once all the regions have been processed, the retained cardiac pulse rate with a minimum difference becomes the good cardiac pulse rate for comparison on a next iteration.

Cardiac pulse rate estimation from source video data

What is disclosed is a system and method for determining whether a subject is in atrial fibrillation. A video is received of a region of exposed skin of a subject. The video is acquired of a region where a videoplethysmographic (VPG) signal can be registered by at least one imaging channel of a video imaging device. For each batch of image frames, pixels associated with the region of exposed skin are isolated and processed to obtain a time-series signal. A VPG signal is extracted from the time-series signal. The power spectral density (PSD) is computed across all frequencies within the VPG signal. A pulse harmonic strength (PHS) is calculated for this VPG signal. The pulse harmonic strength is compared to a discrimination threshold, defined herein. A determination is made whether the subject in the video is in atrial fibrillation or in normal sinus rhythm.

Discriminating between atrial fibrillation and sinus rhythm in physiological signals obtained from video

What is disclosed is a system and method for increasing the accuracy of physiological signals obtained from video of a subject being monitored for a desired physiological function. In one embodiment, image frames of a video are received. Successive batches of image frames are processed. For each batch, pixels associated with an exposed body region of the subject are isolated and processed to obtain a time-series signal. If movement occurred during capture of these image frames that is below a pre-defined threshold level then parameters of a predictive model are updated using this batch's time-series signal. Otherwise, the last updated predictive model is used to generate a predicted time-series signal for this batch. The time-series signal is fused with the predicted time-series signal to obtain a fused time-series signal. The time-series signal for each batch is processed to obtain a physiological signal for the subject corresponding to the physiological function.

Increasing accuracy of a physiological signal obtained from a video of a subject

What is disclosed is a novel system and method for extracting photoplethysmographic (PPG) signals (i.e., cardiac signals) on a continuous basis from signals generated from video images captured of a subject being monitored for cardiac function in a non-contact remote sensing environment. In one embodiment, a time-series signal is received. The time-series signal is generated from video images captured of a region of exposed skin where a PPG signal of a subject of interest can be registered. The time-series signal is then divided into batches for processing, with successive batches having at least a 95% overlap with a previous batch. Each of the batches of time-series signals is processed to obtain a PPG signal from each batch. A mid-point of each of these PPG-signals is stitched together to obtain a continuous PPG signal for the subject. The continuous PPG signal for the subject can then viewed on a display device.

Continuous cardiac pulse rate estimation from multi-channel source video data with mid-point stitching

What is disclosed is a novel video processing system and method wherein a plurality of image frames of a video captured using a video camera with a spatial resolution of (M&#xd7;N) in the (x, y) direction, respectively, and a temporal resolution (T) in frames per unit of time. A first and second magnification factor f, fare selected for spatial enhancement in the (x, y) direction. A third magnification factor fis selected for a desired temporal enhancement in (T). The video data is processed using a dictionary comprising high and low resolution patch cubes which are used to induce spatial and temporal components in the video where no data exists. A high resolution course video Xis generated which has an enhanced spatial resolution of (f*M)&#xd7;(f*N) and an enhanced temporal resolution of (f*T) frames. The course high resolution video is then smoothed, when found required, to generate a smoothed high resolution video.

Growing community of inventors

Rochester, NY, United States of America

Survi Kyal