Google Inc.

Mark Spates, Iv

David Roy Schairer

Vincent Yanton Mo

Zhenguo Guan

Chi Yeung Jonathan Ng

Steven Clark

Prem Kumar

Di Zhu

Sumer Mohammed

Yuzhao Ni

Jun Yang

Lisa Williams

Triona Butler

Di Lin

Cindy Tran

Barnaby John James

Robert Toscano

Lei Zhong

Mackenzie Lee Jacoby

David Matthew Anthony Putzolu

Manit Limlamai

Kyle Lund

Matthew Swartwout

Gaurav Nolkha

Stephen Lanham

Amy Lynn Baldwin

Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other functions that the connected device is capable of performing. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with the connected device. Each graphical control element can be arranged according to a status of the connected device, in order to reflect how the connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

Methods, systems, and apparatus for providing composite graphical assistant interfaces for controlling connected devices

The present disclosure provides systems and methods that perform structure-based access control. In particular, rather than relying upon a user-specific credential scheme, which can require manual sharing of user-specific credentials and/or switching between the multiple accounts to access the particular devices, applications, or services associated with such accounts, the systems and methods of the present disclosure facilitate user credentials to be inherited by or otherwise assigned to a structure identifier associated with a structure (e.g., a home in which the user resides), thereby generating a set of structure credentials. This enables other users in the structure, who may be part of a collaborative user group, to access devices, applications, and/or services using the structure credentials.

Structure-based access control

Remote automated assistant component(s) generate client device notification(s) based on a received IoT state change notification that indicates a change in at least one state associated with at least one IoT device. The generated client device notification(s) can each indicate the change in state associated with the at least one IoT device, and can optionally indicate the at least one IoT device. Further, the remote automated assistant component(s) can identify candidate assistant client devices that are associated with the at least one IoT device, and determine whether each of the one or more of the candidate assistant client device(s) should render a corresponding client device notification. The remote automated assistant component(s) can then transmit a corresponding command to each of the assistant client device(s) it determines should render a corresponding client device notification, where each transmitted command causes the corresponding assistant client device to render the corresponding client device notification.

Generating IoT-based notification(s) and provisioning of command(s) to cause automatic rendering of the IoT-based notification(s) by automated assistant client(s) of client device(s)

Various implementations relate to generating, locally at an assistant client device, specific control commands that, when transmitted to a corresponding smart device, are directly interpretable by the corresponding smart device to effectuate a state change at the corresponding smart device, or at a corresponding additional smart device directly controlled by the corresponding smart device. Various implementations additionally or alternatively relate to utilizing local assistant client devices in discovering, provisioning, and/or registering smart devices for an account of a user.

Control and/or registration of smart devices, locally by an assistant client device

Example aspects of the present disclosure are directed to processing voice commands or utterances. For instance, data indicative of a voice utterance can be received. A device topology representation can be accessed. The device topology representation can define a plurality of smart devices associated with one or more structures. The device topology representation can further define a location of each of the plurality of devices within the associated structures. A transcription of the voice utterance can be determined based at least in part on the device topology representation. One or more selected devices and one or more actions to be performed by the one or more selected devices can be determined based at least in part on the determined transcription and the device topology representation.

Processing voice commands based on device topology

A method for operating a home automation system includes receiving a device modification request from a requesting user device of a requesting user to modify a configurable device property of a requested household device. The requesting user is one of one or more users associated with the household. The method further includes identifying the requested household device based on a home structure mapping the household devices to the corresponding device rooms based on a corresponding location of each household device in the residence. The method also includes determining whether the requesting user has an assigned access level granting access to the requested household device and allowing modification of the one or more configurable device properties of the requested household device. When the requesting user has been assigned an access level, the method includes permitting the modification of the at least one configurable device property of the requested household device.

Home automation system

Growing community of inventors

San Francisco, CA, United States of America

Mark Spates, Iv