Google Inc.

Golnaz Ghiasi

Quoc V Le

Wei Yu

Tsung-Yi Lin

Michael Andrew Goodman

Barret Zoph

Ágoston Weisz

Thang Minh Luong

Ekin Dogus Cubuk

Yifeng Lu

Amin Ghafouri

Sanil Jain

Igor Petrovski

Evgeny Sluzhaev

Oscar Akerlund

Rakesh Shivanna

Vikas Peswani

Mingxing Tan

Diana Avram

Jon Shlens

Komal Singh

Khyatti Gupta

Xiaodan Song

Hieu Hy Pham

Hanxiao Liu

Zihang Dai

Pengchong Jin

Yaguang Li

Yu Qing Du

Xianzhi Du

Yin Cui

Apoorv Kulshreshtha

Dustin Tran

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using memory-optimized contrastive learning to train image encoder and text encoder neural networks.

Memory-optimized contrastive learning

Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.

Multimedia content management for large language model(s) and/or other generative model(s)

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an image representation neural network.

Multi-task self-training for learning general representations

A computer-implemented method of generating scale-permuted models can generate models having improved accuracy and reduced evaluation computational requirements. The method can include defining, by a computing system including one or more computing devices, a search space including a plurality of candidate permutations of a plurality of candidate feature blocks, each of the plurality of candidate feature blocks having a respective scale. The method can include performing, by the computing system, a plurality of search iterations by a search algorithm to select a scale-permuted model from the search space, the scale-permuted model based at least in part on a candidate permutation of the plurality of candidate permutations.

Scale-permuted machine learning architecture

Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

Generating multi-modal response(s) through utilization of large language model(s)

Example aspects of the present disclosure are directed to systems and methods for learning data augmentation strategies for improved object detection model performance. In particular, example aspects of the present disclosure are directed to iterative reinforcement learning approaches in which, at each of a plurality of iterations, a controller model selects a series of one or more augmentation operations to be applied to training images to generate augmented images. For example, the controller model can select the augmentation operations from a defined search space of available operations which can, for example, include operations that augment the training image without modification of the locations of a target object and corresponding bounding shape within the image and/or operations that do modify the locations of the target object and bounding shape within the training image.

Learning data augmentation strategies for object detection

Growing community of inventors

Mountain View, CA, United States of America

Golnaz Ghiasi