Google Inc.

Yu Qing Du

Michael Andrew Goodman

Wei Yu

Ágoston Weisz

Thang Minh Luong

Eric Jang

Golnaz Ghiasi

Yifeng Lu

Amin Ghafouri

Sanil Jain

Rakesh Shivanna

Vikas Peswani

Igor Petrovski

Evgeny Sluzhaev

Oscar Akerlund

Yaguang Li

Daniel Ho

Apoorv Kulshreshtha

Dustin Tran

Mohi Khansari

Implementations disclosed herein relate to mitigating the reality gap through feature-level domain adaptation in training of a vision-based robotic action machine learning (ML) model. Implementations mitigate the reality gap through utilization of embedding consistency losses and/or action consistency losses during training of the action ML model.

Mitigating reality gap through feature-level domain adaptation in training of vision-based robot action model

Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

Generating multi-modal response(s) through utilization of large language model(s)

Growing community of inventors

Berkeley, CA, United States of America

Yu Qing Du