Google Inc.

Other

University of Toronto

Structura Biotechnology Inc.

David James Fleet

Chitwan Saharia

Ali Punjani

William Chan

Jonathan Ho

Marcus Anthony Brubaker

Mohammad Norouzi

Tim Salimans

Jay Ha Whang

Yi Li

Huiwen Chang

Christopher Albert Lee

Haowei Zhang

Saurabh Saxena

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating images. In one aspect, a method includes: receiving an input text prompt including a sequence of text tokens in a natural language; processing the input text prompt using a text encoder neural network to generate a set of contextual embeddings of the input text prompt; and processing the contextual embeddings through a sequence of generative neural networks to generate a final output image that depicts a scene that is described by the input text prompt.

Generating images using sequences of generative neural networks

A method includes receiving training data comprising a plurality of pairs of images. Each pair comprises a noisy image and a denoised version of the noisy image. The method also includes training a multi-task diffusion model to perform a plurality of image-to-image translation tasks, wherein the training comprises iteratively generating a forward diffusion process by predicting, at each iteration in a sequence of iterations and based on a current noisy estimate of the denoised version of the noisy image, noise data for a next noisy estimate of the denoised version of the noisy image, updating, at each iteration, the current noisy estimate to the next noisy estimate by combining the current noisy estimate with the predicted noise data, and determining a reverse diffusion process by inverting the forward diffusion process to predict the denoised version of the noisy image. The method additionally includes providing the trained diffusion model.

Image-to-image mapping by iterative de-noising

Provided are systems and methods for determining 3D structure and 3D motion of a protein molecule from 2D or 3D particle observation images. The method includes: initializing pose parameters and unknown model parameters; performing image formation which includes: generating one or more 3D deformation fields by inputting a latent coordinate vector into the one or more flow generators; performing a convection and projection operation; and performing CTF corruption; fitting the unknown model parameters to the experimental images by gradient-based optimization of an objective function; latent variable search for a given experimental image which includes: performing the image formation one or more times to generate simulated images; and selecting one or more latent coordinate vectors based on similarity; and updating the at least one of the unknown model parameters which includes: generating simulated images; and evaluating the objective function; computing the gradient of the objective function.

Methods and systems for reconstruction of three-dimensional structure and three-dimensional motion of a protein molecule

A method includes receiving, by a computing device, training data comprising a plurality of pairs of images, wherein each pair comprises an image and at least one corresponding target version of the image. The method also includes training a neural network based on the training data to predict an enhanced version of an input image, wherein the training of the neural network comprises applying a forward Gaussian diffusion process that adds Gaussian noise to the at least one corresponding target version of each of the plurality of pairs of images to enable iterative denoising of the input image, wherein the iterative denoising is based on a reverse Markov chain associated with the forward Gaussian diffusion process. The method additionally includes outputting the trained neural network.

Image enhancement via iterative refinement based on machine learning models

There is provided systems and methods for generating 3D structure estimation of at least one target from a set of 2D Cryo-electron microscope particle images. The method includes: receiving the set of 2D particle images of the target from a Cryo-electron microscope; splitting the set of particle images into at least a first half-set and a second half-set; iteratively performing: determining local resolution estimation and local filtering on at least a first half-map associated with the first half-set and a second half-map associated with the second half-set; aligning 2D particles from each of the half-sets using at least one region of the associated half-map; for each of the half-maps, generating an updated half-map using the aligned 2D particles from the associated half-set; and generating a resultant 3D map using all the half-maps.

Methods and systems for 3D structure estimation using non-uniform refinement

Disclosed herein are systems and methods for efficient 3D structure estimation from images of a transmissive object, including cryo-EM images. The method generally comprises, receiving a set of 2D images of a target specimen from an electron microscope, carrying out a reconstruction technique to determine a likely molecular structure, and outputting the estimated 3D structure of the specimen. The described reconstruction technique comprises: establishing a probabilistic model of the target structure; optimizing using stochastic optimization to determine which structure is most likely; and, optionally utilizing importance sampling to minimize computational burden.

Methods and systems for 3D structure estimation

A system and a method for image alignment between at least two images to a three-dimensional model. The method including: determining a lower bound and an upper bound of an acceptable likelihood of mismatch between the at least two images; evaluating the likelihood of mismatch between the at least two images over a set of poses (r), shifts (t), or both poses (r) and shifts (t); and discarding those evaluations resulting beyond the lower bound and upper bound.

Growing community of inventors

Toronto, Canada

David James Fleet