Google Inc.

Kalyana Ram Desineni

Gabriel Taubman

Keun Soo Yim

Ilya Firman

Adam Coimbra

Mugurel Ionut Andreica

Svetoslav Ganov

Cliff Kuang

Mikhail Reutov

Robert Berry

Michael Oh

The present disclosure is directed to automating user interactions in an application. In particular, the computing device can determine, based on the input data, an intent associated with performing an action on the computing device. Additionally, the computing device can invoke, using an application-level assistant component configured to provide an application-level interface on the computing device, execution of an application. Moreover, the computing device can transmit, to a system-level assistant component configured to provide a system-level interface with the computing device, instructions to interface with an actionable component of the application. The instructions can be transmitted using a trusted intermediary component configured to facilitate an interface between the system-level assistant component and the application-level assistant component. Furthermore, the computing device can engage, using the system-level assistant component, the actionable component of the application to perform the action.

Human-in-the-loop voice automation system

Growing community of inventors

San Jose, CA, United States of America

Kalyana Ram Desineni