Object Video Labs, LLC

Samsara Inc.

Other

Alarm.com Incorporated

Sung Chun Lee

Allison Beach

Gang Qian

Sima Taheri

Sravanthi Bondugula

Narayanan Ramanathan

Yan Wang

Nathan Hurst

Kenshiro Nakagawa

Maisie Wang

Kevin Lai

Shirish Nair

Cole Jurden

Rachel Demerly

Jovanna Bubar

Aravindh Ramesh

Justin Levine

Olamide Akintewe

Benjamin Asher Berg

Nafiseh Salmani Niyasar

Sravanti Bondugula

Techniques are presented for detecting when drivers drive while drowsy. In some implementations, a drowsiness model is trained with data associated with inward videos and outward videos captured during a trip. The inward videos capture the inside of the cabin with the driver, and the outward videos capture the view in front of the vehicle in the direction of travel. Further, a device at the vehicle periodically calculates a drowsiness scale index value that indicates the level of drowsiness of the driver. Calculating the drowsiness scale index value includes obtaining a set of inward frames from the inward videos; for each inward frame, creating a face image by cropping the inward frame; obtaining a set of outward frames from the outward videos; calculating inward embeddings of the face images and outward embeddings of the outward frames; and calculating, by the drowsiness model, the drowsiness scale index value.

Drowsy driving detection

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using dual descriptor data. One of the methods includes: detecting, using a first set of descriptor features included in dual descriptor data, a first representation within first image data collected by a camera; determining a change to an imaging modality of the camera; detecting, using a second set of features included in the dual descriptor data, a second representation within second image data collected by the camera; classifying the first representation and the second representation as associated with a same object using the dual descriptor data; and in response to classifying the first representation and the second representation as associated with the same object using the dual descriptor data, transmitting operational instructions to one or more appliances connected to the system.

Dual descriptor data for object recognition in low light conditions

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for reducing camera false detections. One of the methods includes providing, to a neural network of an image classifier that is trained to detect objects of two or more classification types, a feature vector for a respective training image; receiving, from the neural network, an output vector that indicates, for each of the two or more classification types, a likelihood that the respective training image depicts an object of the corresponding classification type; accessing, from two or more ground truth vectors each for one of the two or more classification types, a ground truth vector for the classification type of an object depicted in the training image; and adjusting one or more weights in the neural network using the output vector and the ground truth vector; and storing, in a memory, the image classifier.

Reducing false detections for night vision cameras

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an event detector. The methods, systems, and apparatus include actions of identifying a portion of a first interframe difference image that represents motion of an OI, determining that a second interframe difference image represents motion by a non-OI, combining the portion of the first interframe difference image and the second interframe difference image as a third interframe difference image labeled as motion of both an OI and a non-OI, and training an event detector with the third interframe difference image.

Augmenting training samples for motion detection systems using image additive operations

Methods, systems, an apparatus, including computer programs encoded on a storage device, for training an image classifier. A method includes receiving an image that includes a depiction of an object; generating a set of poorly localized bounding boxes; and generating a set of accurately localized bounding boxes. The method includes training, at a first learning rate and using the poorly localized bounding boxes, an object classifier to classify the object; and training, at a second learning rate that is lower than the first learning rate, and using the accurately localized bounding boxes, the object classifier to classify the object. The method includes receiving a second image that includes a depiction of an object; and providing, to the trained object classifier, the second image. The method includes receiving an indication that the object classifier classified the object in the second image; and performing one or more actions.

Training image classifiers

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using feature descriptors to track objects depicted in images. One of the methods includes receiving hue, saturation, value data for an image and data that indicates an object detected in the image, generating a feature descriptor that includes hue data and saturation data, determining, for each of two or more tracked objects that each have a historical feature descriptor that includes historical hue data and historical saturation data, a distance between (i) the respective historical feature descriptor and (ii) the feature descriptor, associating the feature descriptor for the object with a tracked object from the two or more tracked objects, and tracking the tracked object in one or more images from a video sequence using the feature descriptor and the historical feature descriptor.

Object tracking with feature descriptors

Methods, systems, and apparatus for motion-based human video detection are disclosed. A method includes generating a representation of a difference between two frames of a video; providing, to an object detector, a particular frame of the two frames and the representation of the difference between two frames of the video; receiving an indication that the object detector detected an object in the particular frame; determining that detection of the object in the particular frame was a false positive detection; determining an amount of motion energy where the object was detected in the particular frame; and training the object detector based on penalization of the false positive detection in accordance with the amount of motion energy where the object was detected in the particular frame.

Motion-based human video detection

Methods, systems, an apparatus, including computer programs encoded on a storage device, for tracking human movement in video images. A method includes obtaining a first image of a scene captured by a camera; identifying a bounding box around a human detected in the first image; determining a scale amount that corresponds to a size of the bounding box; obtaining a second image of the scene captured by the camera after the first image was captured; and detecting the human in the second image based on both the first image scaled by the scale amount and the second image scaled by the scale amount. Detecting the human in the second image can include identifying a second scaled bounding box around the human detected in the second image scaled by the scale amount.

Scaled human video tracking

Methods and systems, including computer programs encoded on a storage medium, are described for implementing item monitoring using a doorbell camera. A system generates an input video stream that has image frames corresponding to detection of activity at a property. Timing information is generated for the video stream and includes a timestamp for each image frame of the stream. Using the timing information, the system processes a pre-event image frame that precedes detection of the activity and a post-event image frame that coincides with detection of the activity. An image score is computed with respect to placement of a candidate item at the property in response to processing the pre-event and post-event image frames. The image score is used to determine that a first item was delivered to the property or that a second item was removed after being delivered to the property.

Item monitoring for doorbell cameras

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for tracking objects of interest using distance-based thresholding. One of the methods includes detecting an object depicted in an image captured by a camera, determining a predicted physical distance between the object and the camera, selecting, from a plurality of predetermined confidence thresholds, a confidence threshold for the predicted physical distance, each confidence threshold in the plurality of predetermined confidence thresholds for a different physical distance range, the confidence threshold having a physical distance range that includes the predicted physical distance, and determining, using the confidence threshold and a confidence score that indicates a likelihood that the object is an object of interest, that the object is likely an object of interest.

Object of interest detector distance-based multi-thresholding

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for tracking moving objects depicted in multiple images. One of the methods includes determining, for an image captured by a camera, a first bounding box that represents a first moving object depicted in the image, determining that the first bounding box and a second bounding box overlap in an overlap area, determining that the first moving object represented by the first bounding box was farther from the camera that captured the image than a second moving object represented by the second bounding box, generating a mask for the first bounding box based on the overlap area, and determining, using data from the image that is associated with the mask, that the first moving object matches an appearance of another moving object depicted in another image captured by the camera.

Target association using occlusion analysis, clustering, or both

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for semi-supervised training of an object recognition model. The methods, systems, and apparatus include a monitoring system including a camera located at a property and configured to generate images and one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform actions of determining a cluster of images meets a threshold for number of included images and a threshold for cluster tightness. A representative image of the cluster is selected and a query including the representative image of the cluster is provided. User feedback responsive to the query is received and an object recognition model is updated based on the user feedback.

Growing community of inventors

Tysons, VA, United States of America

Sung Chun Lee