Robert Boschgmbh

Gdm Holding LLC

Emiel Hoogeboom

Max Welling

William Harris Beluch

Priyank Jaini

Dan Zhang

Jorn Peters

Alexandra Lindt

Didrik Nielsen

Anna Khoreva

Mostafa Dehghani

Melih Kandemir

Thomas Andy Keller

Phillip Lippe

Karim Said Mahmoud Barsim

Jakub Tomczak

Jonathan Heek

Kaspar Sakmann

Simon Passenheim

A computer-implemented method of generating multimodal data. The method comprises using a token generation neural network to generate, autoregressively, an output sequence of multimodal tokens, and in response to a next multimodal token being a start-of-image token, generating an image using an image generation subsystem conditioned on features representing the current sequence of multimodal tokens obtained from the token generation neural network. The method further comprises processing the image to convert pixels of the image into a sequence of image tokens, each image token comprising a block encoding of values of the pixels in a different region of the image that maps a set of values of the pixels to a respective image token, and appending the sequence of image tokens to the current output sequence of multimodal tokens as the next multimodal tokens in the output sequence of multimodal tokens.

Systems and methods for generating multimodal data using a single-tower architecture with a data generation subsystem

A computer-implemented method of training an image classifier which uses any combination of labelled and/or unlabelled training images. The image classifier comprises a set of transformations between respective transformation inputs and transformation outputs. An inverse model is defined in which for a deterministic, non-injective transformation of the image classifier, its inverse is approximated by a stochastic inverse transformation. During training, for a given training image, a likelihood contribution for this transformation is determined based on a probability of its transformation inputs being generated by the stochastic inverse transformation given its transformation outputs. This likelihood contribution is used to determine a log-likelihood for the training image to be maximized (and its label, if the training image is labelled), based on which the model parameters are optimized.

Image classifier comprising a non-injective transformation

A computer-implemented method of training an image analysis model. A coupling layer determines an output vector of integer values from an input vector of integer values. The coupling layer is applied by dividing the input vector into non-overlapping first and second input parts; applying a machine learnable submodel of the coupling layer to the first input part to obtain a submodel output of the machine learnable submodel; sampling a transformation vector from a discrete probability distribution, wherein the discrete probability distribution is parameterized based on the submodel output; determining a second output part based on the second input part and the transformation vector; and combining the first input part and the second output part to obtain the output vector. During backpropagation, a gradient of the sampling of the transformation vector is estimated.

Image analysis model comprising an integer coupling layer

A computer-implemented method of training an image analysis model. The image analysis model comprises a coupling layer that determines an output vector of discrete values from an input vector of discrete values. First, a machine learnable submodel of the coupling layer is trained to predict a second input part of the coupling layer from a first input part of the coupling layer. Next, the image analysis model is trained. This involves applying the coupling layer by applying the machine learnable submodel to the first input part to obtain a prediction of the second input part; and determining a second output part by applying an invertible mapping to the second input part defined by the prediction of the second input part. The mapping maps a predicted value of an element of the second input part to a fixed value independent from the predicted value.

Image analysis model comprising a discrete coupling layer

A device and a method for generating a compressed network from a trained neural network are provided. The method includes: a model generating a compressing map from first training data, the compressing map representing the impact of model components of the model to first output data in response to the first training data; generating a compressed network by compressing the trained neural network in accordance with the compressing map; the trained neural network generating trained network output data in response to second training data; the compressed network generating compressed network output data in response to the second training data; training the model by comparing the trained network output data with the compressed network output data.

Device and method for generating a compressed network from a trained neural network

A computer-implemented method of training an image generation model. The image generation model comprises an argmax transformation configured to compute a discrete index feature indicating an index of a feature of the continuous feature vector with an extreme value. The image generation model is trained using a log-likelihood optimization. This involves obtaining a value of the index feature for the training image, sampling values of the continuous feature vector given the value of the index feature according to a stochastic inverse transformation of the argmax transformation, and determining a likelihood contribution of the argmax transformation for the log-likelihood based on a probability that the stochastic inverse transformation generates the values of the continuous feature vector given the value of the index feature.

Image generation model based on log-likelihood

A computer-implemented method for training a normalizing flow. The normalizing flow predicts a first density value based on a first input image. The first density value characterizes a likelihood of the first input image to occur. The first density value is predicted based on an intermediate output of a first convolutional layer of the normalizing flow. The intermediate output is determined based on a plurality of weights of the first convolutional layer. The method for training includes: determining a second input image; determining an output, wherein the output is determined by providing the second input image to the normalizing flow and providing an output of the normalizing flow as output; determining a second density value based on the output tensor and on the plurality of weights; determining a natural gradient of the plurality of weights with respect to the second density value; adapting the weights according to the natural gradient.

Device and method for training a normalizing flow

A device for and a computer implemented method of digital signal processing. The method includes providing a first set of data, mapping the first set of data with to a second set of data, and determining an output of the digital signal processing depending on the second set of data. The second set of data is determined depending on a sum of a finite series of terms. At least one term of the series is determined depending on a result of a convolution of the first set of data with a kernel and at least one term of the series is determined depending on the first set of data and independent of the kernel.

Device for and computer implemented method of digital signal processing

Computer implemented method for digital image data, digital video data or digital audio data enhancement, and a computer implemented method for encoding or decoding this data in particular for transmission or storage, wherein an element representing a part of said digital data comprises an indication of a position of the element in an ordered input data of a plurality of data elements, wherein a plurality of elements is transformed to a representation depending on an invertible linear mapping, wherein the invertible linear mapping maps the input of the plurality of elements to the representation, wherein the invertible linear mapping comprises at least one autoregressive convolution.

Growing community of inventors

Amsterdam, Netherlands

Emiel Hoogeboom