Robert Boschgmbh

Carnegie Mellon University

Wan-Yi Lin

Filipe J Cabrita Condessa

Jeremy Zico Kolter

João D Semedo

Devin T Willmott

Chaithanya Kumar Mummadi

Karren Yang

Yiding Jiang

Aniruddha Saha

S Alireza Golestaneh

Huan Zhang

Manash Pratim

Jan Hendrik Metzen

Jeremy Zieg Kolter

Stefan Gehrer

Filipe Condessa

Fatemeh Sheikholeslami

Mohammad Sadegh Norouzzadeh

Chen Qiu

Chirag Pabbaraju

Shuhua Yu

Joao Semedo

Jianghong Shi

Hariharan Manikandan

Leslie Rice

Akash Umakantha

Aditi Raghunathan

Jinghao Shi

Jeffrey Thompson

Ivan Batalov

Christina Baek

Methods and systems for training a machine learning model with measurement data captured during a manufacturing process. Measurement data regarding a physical characteristic of a plurality of manufactured parts is received as measured by a plurality of sensors at various manufacturing stations. A time-series dynamics machine learning model encodes the measurement data into a latent space having a plurality of nodes. Each node is associated with the measurement data of one of the manufactured parts and at one of the manufacturing stations. A batch of the measurement data can be built, the batch include a first node and a first plurality of nodes immediately connected to the first node via first edges, and measured in time earlier than the first node. A prediction machine learning model can predict measurements of a first of the manufactured parts based on the latent space of the batch of nodes.

Methods and systems for training a machine learning model with measurement data captured during manufacturing process

A system includes a controller configured to generate an original patch utilizing Bayesian optimization, output the original patch at a display at a scene and determine if the original patch does not meet a success criteria of the machine-learning model, in response to the original patch not meeting the success criteria, upscaling the patch, decompose the upscaled patch into o components, for each of the components, utilize Bayesian optimization to update one of the components of the upscaled patch and freezing the other components to generate an updated patch, in response to the updated patch meeting the success criteria, output the updated upscaled patch, and in response to the updated upscaled patch not meeting the success criteria, iteratively update the unfrozen components and determine if the success criteria is met and if not met, unfreeze the frozen components and iteratively update the unfrozen components until the success criteria is met.

System and method for high-resolution blackbox patch attack with Bayesian optimization

A system and method include dividing a source image into a plurality of source regions, which are portions of the source image that correspond to a plurality of grid regions. A mask is used to create a first masked region that masks a first source region and a first unmasked region that comprises a second source region. A first inpainted region is generated by inpainting the first masked region based on the second source region. Similarity data is generated based on a similarity assessment. A protected image is generated that includes at least (i) the first masked region at a first grid region when the similarity data indicates that the first source region is not similar to the first inpainted region and (ii) the first inpainted region at the first grid region when the similarity data indicates that the first source region is similar to the first inpainted region.

System and method with masking and inpainting strategy for generic defense against patch attacks

A computer-implemented method includes converting tabular data to a text representation, generating metadata associated with the text representation of the tabular data, outputting one or more natural language data descriptions indicative of the tabular data in response to utilizing a large language model (LLM) and zero-shot prompting of the metadata and text representation of the tabular data, outputting one or more summaries utilizing the LLM and appending a prompt on the one or more natural language data descriptions, selecting a single summary of the one or more summaries in response to the single summary having a smallest validation rate, receiving a query associated with the tabular data, outputting one or more predictions associated with the query, and in response to meeting a convergence threshold with the one or more predictions generated from the one or more iterations, output a final prediction associated with the query.

System and method for utilizing weak learners on large language models

A computer-implemented system and method include performing neural style transfer augmentations using at least a content image, a first style image, and a second style image. A first augmented image is generated based at least on content of the content image and a first style of the first style image. A second augmented image is generated based at least on the content of the content image and a second style of the second style image. The machine learning system is trained with training data that includes at least the content image, the first augmented image, and the second augmented image. A loss output is computed for the machine learning system. The loss output includes at least a consistency loss that accounts for a predicted label provided by the machine learning system with respect to each of the content image, the first augmented image, and the second augmented image.

System and method for augmenting vision transformers

A method of identifying an attack comprising receiving an input of one or more images, wherein the one or more images includes a patch size and size, divide the image into a first sub-image and a second sub-image, classify the first sub-image and the second sub-image, wherein classifying is accomplished via introducing a variable in a pixel location associated with the first and second sub-image, and in response to classifying the first and second sub-image and identifying an adversarial patch, output a notification indicating that the input is not certified.

Method and system of crown based for adversarial attacks

A computer-implemented system and method relate to certified defense against adversarial patch attacks. A set of one-mask images is generated using a first mask at a set of predetermined regions of a source image. The source image is obtained from a sensor. A set of one-mask predictions is generated, via a machine learning system, based on the set of one-mask images. A first one-mask image is extracted from the set of one-mask images. The first one-mask image is associated with a first one-mask prediction that is identified as a minority amongst the set of one-mask predictions. A set of two-mask images is generated by masking the first one-mask image using a set of second masks. The set of second masks include at least a first submask and a second submask in which a dimension of the first submask is less than a dimension of the first mask. A set of two-mask predictions is generated based on the set of two-mask images. Class data, which classifies the source image, is selected based on the set of two-mask predictions.

System and method with masking for certified defense against adversarial patch attacks

A system includes a machine-learning network. The network includes an input interface configured to receive input data from a sensor. The processor is programmed to receive the input data, generate a perturbed input data set utilize the input data, wherein the perturbed input data set includes perturbations of the input data, denoise the perturbed input data set utilizing a denoiser, wherein the denoiser is configured to generate a denoised data set, send the denoised data set to both a pre-trained classifier and a rejector, wherein the pre-trained classifier is configured to classify the denoised data set and the rejector is configured to reject a classification of the denoised data set, train, utilizing the denoised input data set, the a rejector to achieve a trained rejector, and in response to obtaining the trained rejector, output an abstain classification associated with the input data, wherein the abstain classification is ignored for classification.

System and method for improving robustness of pretrained systems in deep neural networks utilizing randomization and sample rejection

Methods and systems of estimating an accuracy of a neural network on out-of-distribution data. In-distribution accuracies of a plurality of machine learning models trained with in-distribution data are determined. The plurality of machine learning models includes a first model, and a remainder of models. In-distribution agreement is determined between (i) an output of the first machine learning model executed with an in-distribution dataset and (ii) outputs of a remainder of the plurality of machine learning models executed with the in-distribution dataset. The machine learning models are also executed with an unlabeled out-of-distribution dataset, and an out-of-distribution agreement is determined. The in-distribution agreement is compared with the out-of-distribution agreement. Based on a result of the comparison being within a threshold, an accuracy of the first machine learning model on the unlabeled out-of-distribution dataset is estimated based on (i) the in-distribution accuracies, (ii) the in-distribution agreement, and (iii) the out-of-distribution agreement.

Performance of neural networks under distribution shift

A computer-implemented method for training a machine learning network. The method may include receiving an input data, selecting one or more batch samples from the input data, applying a perturbation object onto the one or more batch samples to create a perturbed sample, running the perturbed sample through the machine learning network, updating the perturbation object in response to the function in response to running the perturbed sample, and outputting the perturbation object in response to exceeding a convergence threshold.

System and method for utilizing perturbation in a multimodal environment

Performing semantic segmentation in an absence of labels for one or more semantic classes is provided. One or more weak predictors are utilized to obtain label proposals of novel classes for an original dataset for which at least a subset of sematic classes are unlabeled classes. The label proposals are merged with ground truth of the original dataset to generate a merged dataset, the ground truth defining labeled classes of portions of the original dataset. A machine learning model is trained using the merged dataset. The machine learning model is utilized for performing semantic segmentation on image data.

Learning semantic segmentation models in the absence of a portion of class labels

A method of image segmentation includes receiving one or more images, determining a loss component, for each pixel one image of the one or more images, identifying a majority class and identify a cross-entropy loss between a network output and a target, randomly selecting pixels associated with the one image and select a second set of pixels to compute a super pixel loss for each pair of pixels, summing corresponding loss associated with each pair of pixels, for each corresponding frame of the plurality of frames of the image, computing a flow loss, a negative flow loss, a contrastive optical flow loss, and a equivariant optical flow loss, computing a final loss including a weighted average of the flow loss, the cross entropy loss, the super pixel loss, and foreground loss, updating a network parameter and outputting a trained neural network.

System and method for a hybrid unsupervised semantic segmentation

A multimodal perception system for an autonomous vehicle includes a first sensor that is one of a video, RADAR, LIDAR, or ultrasound sensor, and a controller. The controller may be configured to, receive a first signal from a first sensor, a second signal from a second sensor, and a third signal from a third sensor, extract a first feature vector from the first signal, extract a second feature vector from the second signal, extract a third feature vector from the third signal, determine an odd-one-out vector from the first, second, and third feature vectors via an odd-one-out network of a machine learning network, based on inconsistent modality prediction, fuse the first, second, and third feature vectors and odd-one-out vector into a fused feature vector, output the fused feature vector, and control the autonomous vehicle based on the fused feature vector.

Defending multimodal fusion models against single-source adversaries

A system and method relate to providing machine learning predictions with defenses against patch attacks. The system and method include obtaining a digital image and generating a set of location data via a random process. The set of location data include randomly selected locations on the digital image that provide feasible bases for creating regions for cropping. A set of random crops is generated based on the set of location data. Each crop includes a different region of the digital image as defined in relation to its corresponding location data. The machine learning system is configured to provide a prediction for each crop of the set of random crops and output a set of predictions. The set of predictions is evaluated collectively to determine a majority prediction from among the set of predictions. An output label is generated for the digital image based on the majority prediction. The output label includes the majority prediction as an identifier for the digital image.

Growing community of inventors

Wexford, PA, United States of America

Wan-Yi Lin