Google Inc.

Agilence, Inc.

Wei Hua

Rodrigo Lima Carceroni

Jason Noah Laska

Hrishikesh Balkrishna Aradhye

Ruei-Sung Lin

Lawrence W Neal

Akshay R Bapat

Prateek Reddy

Marius Renn

Pannag Raghunath Sanketi

Juwei Lu

Jonathan J Huang

Jonathon Shlens

George Alban Heitz

Haisong Gu

Amarnag Subramanya

Barret Zoph

Hui Fang

Rizwan Ahmed Chaudhry

Kevin Patrick Murphy

Huazhong Ning

Jia Li

Chenxi Liu

Xiangrong Chen

Fei-Fei Li

Jennifer Ann Gillenwater

Ryan Crabb

Srivatsan Varadharajan

Jonathan Cook

Jinman Kang

Yifan Peng

Mohammed Saberian

Vicente Ignacio Ordonez Roman

Emre Sargin

Mohammad Saberian

Jon Cook

The various embodiments described herein include methods, devices, and systems for categorizing motion events. In one aspect, a method is performed at a camera device. The method includes: (1) capturing a plurality of video frames via the image sensor, the plurality of video frames corresponding to a scene in a field of view of the camera; (2) sending the video frames to the remote server system in real-time; (3) while sending the video frames to the remote server system in real-time: (a) determining that motion has occurred within the scene; (b) in response to determining that motion has occurred within the scene, characterizing the motion as a motion event; and (c) generating motion event metadata for the motion event; and (4) sending the generated motion event metadata to the remote server system concurrently with the video frames.

Systems and methods for categorizing motion events

A method for determining an architecture for a task neural network configured to perform a particular machine learning task is described. The method includes obtaining data specifying a current set of candidate architectures for the task neural network; for each candidate architecture in the current set: processing the data specifying the candidate architecture using a performance prediction neural network having multiple performance prediction parameters, the performance prediction neural network being configured to process the data specifying the candidate architecture in accordance with current values of the performance prediction parameters to generate a performance prediction that characterizes how well a neural network having the candidate architecture would perform after being trained on the particular machine learning task; and generating an updated set of candidate architectures by selecting one or more of the candidate architectures in the current set based on the performance predictions for the candidate architectures in the current set.

Neural architecture search using a performance prediction neural network

Certain embodiments of the disclosed technology include systems and methods for determining the priority of a notification on a mobile device using machine learning. Other aspects of the disclosed technology include selectively displaying notifications based on the priority of a notification. According to an embodiment of the disclosed technology, a computer-implement method is provided that comprises outputting, to a display device operatively coupled to a mobile device, a plurality of notifications, wherein each respective notification from the plurality of notifications is associated with a respective priority score; modifying, by the mobile device, a ranking model based on a user input received responsive to a first notification from the plurality of notifications and a characteristic of a second notification from the plurality of notifications; determining, by the mobile device, a priority score associated with a third notification based on the modified ranking model; and outputting, to the display device, the third notification based on the priority score associated with the third notification, wherein the third notification is graphically emphasized responsive to the priority score associated with the third notification being greater than at least one respective priority score associated with a corresponding respective notification from the plurality of notifications.

Systems and methods for prioritizing notifications on mobile devices

The various embodiments described herein include methods, devices, and systems for categorizing motion events. In one aspect, a method includes: (1) obtaining a plurality of video frames, the plurality of video frames corresponding to a scene and a motion event candidate; (2) identifying one or more visual characteristics of the scene; (3) obtaining one or more background factors for the scene; (4) utilizing the obtained background factors to identify one or more motion entities; (5) for each identified motion entity: (a) classifying the motion entity by performing object recognition; and (b) obtaining one or more representative motion vectors based on a motion track of the motion entity; and (6) assigning a motion event category to the motion event candidate based on the identified visual characteristics, the obtained background factors, the classified motion entities, and the obtained representative motion vectors.

The various embodiments described herein include methods, devices, and systems for analyzing video streams. In one aspect, a method includes, while receiving a video stream: obtaining motion start information indicating that a portion of the video stream includes a motion event candidate; and segmenting the portion of the video stream into a plurality of segments including an initial segment. The method also includes obtaining a first categorization for the motion event candidate based on the initial segment; and, in accordance with the obtained first categorization, generating a log entry for the motion event candidate including the first categorization. The method further includes: in response to obtaining motion end information, obtaining a second categorization for the motion event based on the plurality of segments; and updating the log entry for the motion event candidate based on the obtained second categorization.

Systems and methods for analyzing a video stream

The disclosed technology includes techniques for improved content coverage in automatically-generated content summaries. The technique may include clustering a set of input content, determining diffusion for each cluster, and selecting representatives of each cluster to optimize other secondary metrics. Various types of input content may be used, including groups of images, video clips, or other multimedia content. Contiguous content may be manually or programmatically divided into discrete portions before clustering, for example, a lengthy video divided into a number of short clips. In some implementations, the disclosed technique may be implemented effectively on a mobile device. In other words, the processing required may be computationally feasible for execution on a smartphone or similar device.

Systems and methods for improved coverage of input media in content summarization

The various embodiments described herein include methods, devices, and systems for categorizing motion event candidates. In one aspect, a method includes receiving and processing video frames that include a motion event candidate. The processing includes: (a) obtaining background factors corresponding to a background in at least a subset of the video frames; (b) utilizing the background factors to identify one or more motion entities; (c) for each motion entity, obtaining one or more representative motion vectors based on a motion track of the respective motion entity; (d) identifying one or more features in at least a subset of the video frames; and (e) aggregating the background factors, the representative motion vectors, and the features to generate motion features. The method further includes sending the motion features to an event categorizer, where the event categorizer assigns a motion event category to the motion event candidate based on the received motion features.

Systems and methods for categorizing motion event candidates

Techniques for determining motion saliency in video content using center-surround receptive fields. In some implementations, images or frames from a video may be apportioned into non-overlapped regions, for example, by applying a rectilinear grid. For each grid region, or cell, motion consistency may be measured between the center and surround area of that cell across frames of the video. Consistent motion across the center-surround area may indicate that the corresponding region has low variation. The larger the difference between center-surround motions in a cell, the more likely the region has high motion saliency.

Systems and methods for determining motion saliency

A computer-implemented technique can receive, at a computing device including one or more processors, a plurality of photos. The technique can extract quality features and similarity features for each of the plurality of photos and can obtain weights for the various quality features and similarity features based on an analysis of a reference photo collection. The technique can generate a quality metric for each of the plurality of photos and can generate a similarity matrix for the plurality of photos by analyzing the various quality features and similarity features and using the obtained weights. The technique can perform joint global maximization of photo quality and photo diversity using the quality metrics and the similarity matrix in order to select a subset of the plurality of photos having a high degree of representativeness. The technique can then store the subset of the plurality of photos in a memory.

Techniques for automatic photo album generation

Disclosed are methods for automatically generating commands to transform a video sequence based on information regarding speaking participants derived from the audio and video signals. The audio stream is analyzed to detect individual speakers and the video is optionally analyzed to detect lip movement to determine a probability that a detected participant is speaking. Commands are then generated to transform the video stream consistent with the identified speaker.

Method and apparatus for focus-of-attention control

A method and apparatus for virtual camera operation is disclosed. Virtual camera operation may include identifying potential subjects of a video stream by identifying faces of participants in the input video stream. Virtual camera operation may include determining a speaking state of each participant in the input video stream based on their respective identified face. Virtual camera operation may include identifying a subject of the input video stream based on the speaking state. Virtual camera operation may include generating, using a processor, an output video stream including a portion of the input video stream based on the subject.

Virtual camera operator

A method for assessing events detected by a surveillance system includes assessing the likelihood that the events correspond to events being monitored from feedback in response to a condition set by a user. Classifiers are created for the events from the feedback. The classifiers are applied to allow the surveillance system improve its accuracy when processing new video data.

Active and adaptive intelligent video surveillance system

Implementations relate to a computer-implemented method and a device for determining a relative posed between devices. The method can include receiving data representing first keypoint features from a first image captured by a camera of a second mobile computing device; capturing, by a camera of a first mobile computing device, a second image, wherein the first image and the second image comprise a substantially common scene having an area of overlap; computing, by the first mobile computing device, data representing second keypoint features from the second image; determining, by the first mobile computing device, based at least in part on the data representing first keypoint features and the data representing second keypoint features, a relative pose of the first mobile computing device and the second mobile computing device; and communicating the relative pose to the second mobile computing device.

Growing community of inventors

Palo Alto, CA, United States of America

Wei Hua