Google Inc.

Adrian Zumbrunnen

Zaheed Sabur

Lucas Mirelmann

Richard Ngo

Justin Reid

Kerry Young

Ibrahim Badr

Aparna Rajan Ramchandran

Woonji Kim

Jonathan David Lee

Vladimir Vuskovic

Adriana Teresa Olmos Antillon

Minji Hong

Syeda Saba Zehra Zaidi

Behshad Behzadi

Daniel Cotting

Denis Burakov

Sergey Nazarov

Steve Cheng

Michael Golikov

Bohdan Vlasyuk

Mario Bertschler

Yoo Kyoung Noh

Lucia Terrenghi

Andrew David Swank

Pareezad Cyrus Zarolia

Gokhan Hasan Bakir

Roland Peter Kehl

Adam Grabowski

Alexander Edward Stillwell

Thomas Jonathan Stone

Matthew Ryan Casey

Peter John Hodgson

Andrea Terwisscha Van Scheltinga

Ye Jeong Kim

Rick Jones

Tak Keung Cheng

Zhi Chen

Mikhail Reutov

Andrew Gugel

Apama Rajan Ramchandran

Xinran Lu

Kimberly Harvey

James O'Leary

J T DiMartile

Daniel Kamhaji

Kaan Yucer

Erin Lynch

Implementations can reduce the time required to obtain responses from an automated assistant by, for example, obviating the need to provide an explicit invocation to the automated assistant, such as by saying a hot-word/phrase or performing a specific user input, prior to speaking a command or query. In addition, the automated assistant can optionally receive, understand, and/or respond to the command or query without communicating with a server, thereby further reducing the time in which a response can be provided. Implementations only selectively initiate on-device speech recognition responsive to determining one or more condition(s) are satisfied. Further, in some implementations, on-device NLU, on-device fulfillment, and/or resulting execution occur only responsive to determining, based on recognized text form the on-device speech recognition, that such further processing should occur. Thus, through selective activation of on-device speech processing, and/or selective activation of on-device NLU and/or on-device fulfillment, various client device resources are conserved.

Selectively activating on-device speech recognition, and using recognized text in selectively activating on-device NLU and/or on-device fulfillment

Methods, apparatus, and computer readable media are described related to automated assistants that proactively incorporate, into human-to-computer dialog sessions, unsolicited content of potential interest to a user. In various implementations, in an existing human-to-computer dialog session between a user and an automated assistant, it may be determined that the automated assistant has responded to all natural language input received from the user. Based on characteristic(s) of the user, information of potential interest to the user or action(s) of potential interest to the user may be identified. Unsolicited content indicative of the information of potential interest to the user or the action(s) may be generated and incorporated by the automated assistant into the existing human-to-computer dialog session. In various implementations, the incorporating may be performed in response to the determining that the automated assistant has responded to all natural language input received from the user during the human-to-computer dialog session.

Proactive incorporation of unsolicited content into human-to-computer dialogs

Methods, systems, and apparatus for recognizing objects and providing content related to the recognized objects are described. In one aspect, a method includes detecting presence of one or more objects depicted in a viewfinder of a camera of the mobile device. In response to detecting the presence of the one or more objects, image data representing the one or more objects is sent to a content distribution system that selects content related to objects depicted in images. A location of each of the one or more objects in the viewfinder of the camera is tracked while waiting to receive content from the content distribution system. Content related to the one or more objects is received from the content distribution system. A current location of each object in the viewfinder is determined and the content related to the object is presented within the viewfinder at the current location of the object.

Real time object detection and tracking

Implementations set forth herein relate to an automated assistant that can be invoked while a user is interfacing with a foreground application in order to retrieve data from one or more different applications, and then provide the retrieved data to the foreground application. A user can invoke the automated assistant while operating the foreground application by providing a spoken utterance, and the automated assistant can select one or more other applications to query based on content of the spoken utterance. Application data collected by the automated assistant from the one or more other applications can then be used to provide an input to the foreground application. In this way, the user can bypass switching between applications in the foreground in order to retrieve data that has been generated by other applications.

Automated assistant for generating, in response to a request from a user, application input content using application data from other sources

Display screen with animated graphical user interface

Display screen with transitional graphical user interface

Growing community of inventors

Zurich, Switzerland

Adrian Zumbrunnen