Google Inc.

Other

Apple Inc.

Tomas Jon Pfister

Sercan Omer Arik

Jinsung Yoon

Zizhao Zhang

Russell Y Webb

Abhishek A Sharma

Rui Zhang

Jerremy Holland

Nicholas Elia Apostoloff

Jia Li

Siddharth Khullar

Tai-Peng Tian

Chen Xing

Han Zhang

Cuneyt Oncel Tuzel

Joshua M Susskind

Hanjun Dai

Mingfei Gao

Ruoxi Sun

Ashish Shrivastava

Kihyuk Sohn

Chun-Liang Li

Long Zhao

Xingchen Wan

Aspects of the disclosure are directed to automatically selecting examples in a prompt for an LLM to demonstrate how to perform tasks. Aspects of the disclosure can select and build a set of examples from LLM zero-shot outputs via predetermined criteria that can combine consistency, diversity, and repetition. In the zero-shot setting for three different LLMs, using only LLM predictions, aspects of the disclosure can improve performance up to 15% compared to zero-shot baselines and can match or exceed few-shot base-lines for a range of reasoning tasks.

Self-improving LLMs through consistency-based self-generated demonstrations

A method includes receiving image data including a series of image patches of an image. The method includes generating, using a first set of transformers of a vision transformer (V-T) model, a first set of higher order feature representations based on the series of image patches and aggregating the first set of higher order feature representations into a second set of higher order feature representations that is smaller than the first set. The method includes generating, using a second set of transformers of the V-T model, a third set of higher order feature representations based on the second set of higher order feature representations and aggregating the third set of higher order feature representations into a fourth set of higher order feature representations that is smaller than the third set. The method includes generating, using the V-T model, an image classification of the image based on the fourth set.

Aggregating nested vision transformers

A method for active learning includes obtaining a set of unlabeled training samples and for each unlabeled training sample, perturbing the unlabeled training sample to generate an augmented training sample. The method includes generating, using a machine learning model, a predicted label for both samples and determining an inconsistency value for the unlabeled training sample that represents variance between the predicted labels for the unlabeled and augmented training samples. The method includes sorting the unlabeled training samples based on the inconsistency values and obtaining, for a threshold number of samples selected from the sorted unlabeled training samples, a ground truth label. The method includes selecting a current set of labeled training samples including each selected unlabeled training samples paired with the corresponding ground truth label. The method includes training, using the current set and a proper subset of unlabeled training samples, the machine learning model.

Active learning via a sample consistency assessment

A method for training a locally interpretable model includes obtaining a set of training samples and training a black-box model using the set of training samples. The method also includes generating, using the trained black-box model and the set of training samples, a set of auxiliary training samples and training a baseline interpretable model using the set of auxiliary training samples. The method also includes training, using the set of auxiliary training samples and baseline interpretable model, an instance-wise weight estimator model. For each auxiliary training sample in the set of auxiliary training samples, the method also includes determining, using the trained instance-wise weight estimator model, a selection probability for the auxiliary training sample. The method also includes selecting, based on the selection probabilities, a subset of auxiliary training samples and training the locally interpretable model using the subset of auxiliary training samples.

Reinforcement learning based locally interpretable models

A method includes obtaining a batch of training samples. For each particular training sample in the batch of training samples, the method includes generating, using a data value estimator model and the particular training sample, a corresponding predicted value of the particular training sample when used to train a machine learning model. The method includes selecting, based on the corresponding predicted values, a subset of the batch of training samples. For each particular training sample in the subset of the batch of training samples, the method includes determining, using the machine learning model and the particular training sample, a corresponding prediction performance measurement. The method includes adjusting one or more estimator parameter values of the data value estimator model based on the corresponding prediction performance measurements.

Data valuation using reinforcement learning

A method includes receiving a training data set including a plurality of training data subsets. From two or more training data subsets in the training data set, the method includes selecting a support set of training examples and a query set of training examples. The method includes determining, using the classification model, a centroid value for each respective class. For each training example in the query set of training examples, the method includes generating, using the classification model, a query encoding, determining a class distance measure, determining a ground-truth distance, and updating parameters of the classification model. For each training example in the query set of training examples identified as being misclassified, the method further includes generating a standard deviation value, sampling a new query, and updating parameters of the confidence model based on the new query encoding.

Distance-based learning confidence model

A method of interpreting tabular data includes receiving, at a deep tabular data learning network (TabNet) executing on data processing hardware, a set of features. For each of multiple sequential processing steps, the method also includes: selecting, using a sparse mask of the TabNet, a subset of relevant features of the set of features; processing using a feature transformer of the TabNet, the subset of relevant features to generate a decision step output and information for a next processing step in the multiple sequential processing steps; and providing the information to the next processing step. The method also includes determining a final decision output by aggregating the decision step outputs generated for the multiple sequential processing steps.

Interpretable tabular data learning using sequential sparse attention

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing an input data element to generate a prediction output that characterizes the input data element. In one aspect, a method comprises: determining a respective attention weight between an input data element and each of a plurality of reference data elements; processing each of the reference data elements using the encoder neural network to generate a respective value embedding of each reference data element; determining a combined value embedding of the reference data elements based on (i) the respective value embedding of each reference data element, and (ii) the respective attention weight between the input data element and each reference data element; and processing the combined value embedding of the reference data elements using a prediction neural network to generate the prediction output that characterizes the input data element.

Attention-based prototypical learning

A method for training a machine learning model includes obtaining a set of training samples. For each training sample in the set of training samples, during each of one or more training iterations, the method includes cropping the training sample to generate a first cropped image, cropping the training sample to generate a second cropped image that is different than the first cropped image, and duplicating a first portion of the second cropped image. The method also includes overlaying the duplicated first portion of the second cropped image on a second portion of the second cropped image to form an augmented second cropped image. The first portion is different than the second portion. The method also includes training the machine learning model with the first cropped image and the augmented second cropped image.

Self-supervised learning for anomaly detection and localization

A method includes obtaining a source training dataset that includes a plurality of source training images and obtaining a target training dataset that includes a plurality of target training images. For each source training image, the method includes translating, using the forward generator neural network G, the source training image to a respective translated target image according to current values of forward generator parameters. For each target training image, the method includes translating, using a backward generator neural network F, the target training image to a respective translated source image according to current values of backward generator parameters. The method also includes training the forward generator neural network G jointly with the backward generator neural network F by adjusting the current values of the forward generator parameters and the backward generator parameters to optimize an objective function.

Training image-to-image translation neural networks

A method includes obtaining a set of training samples. During each of a plurality of training iterations, the method also includes sampling a batch of training samples from the set of training samples. The method includes, for each training sample in the batch of training samples, determining, using a data value estimator, a selection probability. The selection probability for the training sample is based on estimator parameter values of the data value estimator. The method also includes selecting, based on the selection probabilities of each training sample, a subset of training samples from the batch of training samples, and determining, using a predictor model with the subset of training samples, performance measurements. The method also includes adjusting model parameter values of the predictor model based on the performance measurements, and updating the estimator parameter values of the data value estimator based on the performance measurements.

A method for jointly training a classification model and a confidence model. The method includes receiving a training data set including a plurality of training data subsets. From two or more training data subsets in the training data set, the method includes selecting a support set of training examples and a query set of training examples. The method includes determining, using the classification model, a centroid value for each respective class. For each training example in the query set of training examples, the method includes generating, using the classification model, a query encoding, determining a class distance measure, determining a ground-truth distance, and updating parameters of the classification model. For each training example in the query set of training examples identified as being misclassified, the method further includes generating a standard deviation value, sampling a new query, and updating parameters of the confidence model based on the new query encoding.

A generative network may be learned in an adversarial setting with a goal of modifying synthetic data such that a discriminative network may not be able to reliably tell the difference between refined synthetic data and real data. The generative network and discriminative network may work together to learn how to produce more realistic synthetic data with reduced computational cost. The generative network may iteratively learn a function that synthetic data with a goal of generating refined synthetic data that is more difficult for the discriminative network to differentiate from real data, while the discriminative network may be configured to iteratively learn a function that classifies data as either synthetic or real. Over multiple iterations, the generative network may learn to refine the synthetic data to produce refined synthetic data on which other machine learning models may be trained.

Generating more realistic synthetic data with adversarial nets

Modifying operation of an intelligent agent in response to facial expressions and/or emotions.

Growing community of inventors

Foster City, CA, United States of America

Tomas Jon Pfister