Adobe, Inc.

Yeda Research and Develpment & Co. Ltd

Shai Bagon

Elya Shechtman

Aseem Agarwala

Michal Irani

Omer Bar Tal

Oded Shahar

Daniel Glasner

Tali Dekel

Narek Tumanyan

Using a pre-trained and fixed Vision Transformer (ViT) model as an external semantic prior, a generator is trained given only a single structure/appearance image pair as input. Given two input images, a source structure image and a target appearance image, a new image is generated by the generator in which the structure of the source image is preserved, while the visual appearance of the target image is transferred in a semantically aware manner, so that objects in the structure image are 'painted' with the visual appearance of semantically related objects in the appearance image. A self-supervised, pre-trained ViT model, such as a DINO-VIT model, is leveraged as an external semantic prior, allowing for training of the generator only on a single input image pair, without any additional information (e.g., segmentation/correspondences), and without adversarial training. The method may generate high quality results in high resolution (e.g., HD).

Method and system for semantic appearance transfer using splicing ViT features

A method, system, and computer-readable storage medium for performing content based transitions between images. Image content within each image of a set of images are analyzed to determine at least one respective characteristic metric for each image. A respective transition score for each pair of at least a subset of the images is determined with respect to each transition effect of a plurality of transition effects based on the at least one respective characteristic metric for each image. Transition effects implementing transitions between successive images for a sequence of the images are determined based on the transition scores. An indication of the determined transition effects is stored. The determined transition effects are useable to present the images in a slideshow or other image sequence presentation.

Content aware slideshows

A method implementable on a computing device includes exploiting data redundancy to combine high frequency information from at least two different scales of an input signal to generate a super resolution version of said input signal. An alternative method includes exploiting recurrence of data from an input signal in at least two different scales of at least one reference signal to extract and to combine high frequency information from a plurality of scales of said at least one reference signal to generate a super resolution version of said input signal. An alternative method includes generating a super resolution version of a single input video sequence in at least the temporal dimension by exploiting data recurrence within the input video sequence or with respect to an external database of example video sequences. A signal may be an image, a video sequence, an audio signal, etc.

Super resolution from a single signal

Growing community of inventors

Rehovot, Israel

Shai Bagon