Google Inc.

Carnegie Mellon University

Utsav Prabhu

Andrew C Gallagher

Caroline Rebecca Pantofaru

Vinay Bettadapura

Tuan Anh Nguyen

Jaclyn Konzelmann

Marios Savvides

Michael Nechyba

Hang Chu

Andrew Gallagher

Caroline Pantofaru

Implementations relate to an automated assistant that provides and manages output from one or more elements of output hardware of a computing device. The automated assistant manages dynamic adjustment of access permissions to the computing device according to, for example, a detected presence of one or more users. An active-user queue can be established each time a unique user enters a viewing window of a camera of the computing device when, up to that point, no user was considered active. Multiple image frames can be captured via the camera and processed to determine whether an initial user remains in the viewing window and/or whether another user has entered the viewing window. The initial user can be considered active as long as they are exclusively detected in the viewing window. Restricted content associated with the user may be rendered by the computing device whilst the user is active.

Assigning priority for an automated assistant according to a dynamic user queue and/or multi-modality presence detection

Implementations set forth herein relate to an automated assistant that uses circumstantial condition data, generated based on circumstantial conditions of an input, to determine whether the input should affect an action been initialized by a particular user. The automated assistant can allow each user to manipulate their respective ongoing action without necessitating interruptions for soliciting explicit user authentication. For example, when an individual in a group of persons interacts with the automated assistant to initialize or affect a particular ongoing action, the automated assistant can generate data that correlates that individual to the particular ongoing action. The data can be generated using a variety of different input modalities, which can be dynamically selected based on changing circumstances of the individual. Therefore, different sets of input modalities can be processed each time a user provides an input for modifying an ongoing action and/or initializing another action.

Dynamically assigning multi-modality circumstantial data to assistant action requests for correlating with subsequent requests

Assigning priority for an automated assistant according to a dynamic user queue and/or multimodality presence detection

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for partitioning videos. In one aspect, a method includes obtaining a partition of a video into one or more shots. Features are generated for each shot, including visual features and audio features. The generated features for each shot are provided as input to a partitioning neural network that is configured to process the generated features to generate a partitioning neural network output. The partition of the video into one or more chapters is determined based on the partitioning neural network output, where a chapter is a sequence of consecutive shots that are determined to be taken at one or more locations that are semantically related.

Partitioning videos

A method of dynamically scoring implicit interactions can include receiving, by an interaction analysis server from an imaging system, a plurality of images of an environment captured in a period of time corresponding to display of a presentation, retrieving, by the interaction analysis server, content information corresponding to content of the presentation, and identifying, by a presence detector of the interaction analysis server, that a face appears in at least one image of the plurality of images. The method can further include matching, by a facial recognition system of the interaction analysis server, the face with a user identifier, retrieving, by a client device information retriever, client device information associated with the user identifier and corresponding to the period of time, and calculating, by a client device interaction score calculator, a client device interaction score based on one or more correspondences between the client device information and the content information.

Systems and methods for dynamically scoring implicit user interaction

A novel, unified technique for analyzing, synthesizing, and recognizing faces captured by various modalities, and under a variety of often seen real-world circumstances, using thin-plate splines for densification of points of a face from an initial set of fiducial points.

System and method for face recognition robust to multiple degradations

A system and method for providing a fast, efficient and accurate method for recovery of depth information from a single image of a face is disclosed. The technique uses a novel thin-plate spline-based dense-correspondence method to align the face, and the representation incorporates a weighted framework, interpreting the depth recovery problem as a weighted data problem.

Growing community of inventors

Pittsburgh, PA, United States of America

Utsav Prabhu