Google Inc.

Akshay R Bapat

George Alban Heitz

James Edward Stewart

Joe Delone Venters

Navneet Dalal

Rizwan Ahmed Chaudhry

Yohannes Kifle

Kara Gates

Jennifer Bush

Anton Bastov

Timothy Butler

Jason Noah Laska

Lawrence W Neal

Wei Hua

Prateek Reddy

Mehul Nariyawala

A method for recognizing persons in video streams includes obtaining a live video stream, detecting a first person in the live video stream, determining from analysis of the live video stream first information that identifies an attribute of the first person, determining based on at least some of the first information that the first person is not identifiable to the computing system, storing at least some of the first information, receiving a user classification of the first person as being a stranger, and deleting the stored first information.

Systems and methods of person recognition in video streams

The various embodiments described herein include methods, devices, and systems for categorizing motion events. In one aspect, a method is performed at a camera device. The method includes: (1) capturing a plurality of video frames via the image sensor, the plurality of video frames corresponding to a scene in a field of view of the camera; (2) sending the video frames to the remote server system in real-time; (3) while sending the video frames to the remote server system in real-time: (a) determining that motion has occurred within the scene; (b) in response to determining that motion has occurred within the scene, characterizing the motion as a motion event; and (c) generating motion event metadata for the motion event; and (4) sending the generated motion event metadata to the remote server system concurrently with the video frames.

Systems and methods for categorizing motion events

The various implementations described herein include systems and methods for recognizing persons in video streams. In one aspect, a method includes: obtaining images collected by video cameras in a smart home environment, each image including a detected person; for each image, obtaining first information that identifies an attribute of the detected person, the first information generated from analysis of the image; grouping the images into a first group of a plurality of groups based on the first information, each group of the plurality of groups representing a unique person; receiving a request to remove a first image from the first group; removing the first image from the first group; and disassociating the first information from the first group.

A computing system obtains a first category for a first motion event. The system sends a first alert indicative of the first category to a user. After sending the first alert, it obtains a second category for a second motion event. In accordance with a determination that the second category is the same as the first category, the system determines whether a third motion event of the first category has been detected in a preceding predetermined amount of time before the second motion event. If the third motion event has not been detected in the preceding predetermined amount of time before the second motion event, the system sends a second alert associated with the second motion event indicative of the first category to the user. If the third motion event has been detected in the preceding predetermined amount of time before the second motion event, the system forgoes sending the second alert.

Methods and systems for providing event alerts

The various implementations described herein include systems and methods for recognizing persons in video streams. In one aspect, a method includes: (1) obtaining a live video stream; (2) detecting person(s) in the stream; and (3) determining, from analysis of the live video stream, first information of the detected person(s); (4) determining, based on the first information, that the first person is not known to the computing system; (5) in accordance with the determination that the first person is not known: (a) storing the first information; and (b) requesting a user to classify the first person; and (6) in accordance with a determination that a response was received classifying the first person as a stranger, deleting the stored first information.

The various implementations described herein include systems and methods for recognizing persons in video streams. In one aspect, a method includes: (1) obtaining images collected by video cameras in a smart home environment, each image including a detected person; (2) for each image, obtaining personally identifiable information of the detected person, the personally identifiable information generated from analysis of the image; (3) grouping the images into a first group of a plurality of groups based on the personally identifiable information, each group of the plurality of groups representing a unique person; (4) receiving from a user a request to remove a first image from the first group; and (5) in response to the request: (a) removing the first image from the first group; and (b) disassociating the corresponding personally identifiable information from the first group.

Systems and methods for person recognition data management

The various implementations described herein include systems and methods for recognizing persons in video streams. In one aspect, a method includes: (1) obtaining a live video stream; (2) detecting person(s) in the stream; and (3) determining, from analysis of the live video stream, personally identifiable information of the detected person(s); (4) determining, based on the personally identifiable information, that the first person is not known to the computing system; (5) in accordance with the determination that the first person is not known: (a) storing the personally identifiable information; and (b) requesting a user to classify the first person; and (6) in accordance with (i) a determination that a predetermined amount of time has elapsed since the request was transmitted and a response was not received, or (ii) a determination that a response was received classifying the first person as a stranger, deleting the stored personally identifiable information.

The various embodiments described herein include methods, devices, and systems for providing event alerts. In one aspect, a method includes: (1) receiving a plurality of video frames from a camera, the plurality of video frames including a motion event candidate; (2) categorizing the motion event candidate by processing the plurality of video frames, the categorizing including: (a) associating the motion event candidate with a first category of a plurality of motion event categories; and (b) generating a confidence level for the association of the motion event candidate with the first category; and (3) sending an alert indicative of the first category and the confidence level to a user associated with the camera.

Methods and systems for providing intelligent alerts for events

The various embodiments described herein include methods, devices, and systems for categorizing motion events. In one aspect, a method includes: (1) obtaining a plurality of video frames, the plurality of video frames corresponding to a scene and a motion event candidate; (2) identifying one or more visual characteristics of the scene; (3) obtaining one or more background factors for the scene; (4) utilizing the obtained background factors to identify one or more motion entities; (5) for each identified motion entity: (a) classifying the motion entity by performing object recognition; and (b) obtaining one or more representative motion vectors based on a motion track of the motion entity; and (6) assigning a motion event category to the motion event candidate based on the identified visual characteristics, the obtained background factors, the classified motion entities, and the obtained representative motion vectors.

The various embodiments described herein include methods, devices, and systems for analyzing video streams. In one aspect, a method includes, while receiving a video stream: obtaining motion start information indicating that a portion of the video stream includes a motion event candidate; and segmenting the portion of the video stream into a plurality of segments including an initial segment. The method also includes obtaining a first categorization for the motion event candidate based on the initial segment; and, in accordance with the obtained first categorization, generating a log entry for the motion event candidate including the first categorization. The method further includes: in response to obtaining motion end information, obtaining a second categorization for the motion event based on the plurality of segments; and updating the log entry for the motion event candidate based on the obtained second categorization.

Growing community of inventors

Mountain View, CA, United States of America

Akshay R Bapat