Google Inc.

Diana Avram

Michael Andrew Goodman

Wei Yu

Ágoston Weisz

Thang Minh Luong

Cliff Kuang

Golnaz Ghiasi

Radu Voroneanu

Yifeng Lu

Amin Ghafouri

Mugurel-Ionut Andreica

Sanil Jain

Andre Elisseeff

Dana Ritter

Igor Petrovski

Evgeny Sluzhaev

Oscar Akerlund

Rakesh Shivanna

Vikas Peswani

Komal Singh

Khyatti Gupta

Alice Liang

Adam Coimbra

Sneha Ashok

Kyunghoon Lee

Deepak Goyal

Anton Berezin

Andrea D'olimpio

Florian Hasler

Implementations relate to identifying actions performed by a user while the user is interacting with multiple applications and providing a routine suggestion to the user based on the identified actions. While a user is interacting with one or more applications, the user actions are determined. The user actions are compared to one or more template routines and a template routine is selected that matches the actions of the user and provided to the user as a suggested routine. The suggested routine can be implemented by an automated assistant to perform the actions of the template by providing a corresponding command.

Automatically suggesting routines based on detected user actions via multiple applications

Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.

Multimedia content management for large language model(s) and/or other generative model(s)

Implementations relate to determining a rendering type for an application that is executing automatically. Based on user interactions with an application that is associated with specified input from the user while the user is interacting with the application, a confidence metric is generated for each specified input and a rendering type is determined based on the confidence metrics. Subsequently, when the user requests that a sequence of actions be performed, the application will be displayed according to the rendering type.

Automatically executing application routines with user inputs

Growing community of inventors

Zurich, Switzerland

Diana Avram