Disney Enterprises, Inc.

Eth Zurich

Simone Meyer

Alexander Sorkine Hornung

Christopher Richard Schroers

Abdelaziz Djelouah

Oliver Wang

Markus H Gross

Markus Gross

Brian McWilliams

Henning Zimmer

Max Grosse

Victor Cornillere

A video processing system includes a computing platform having a hardware processor and a memory storing a software code including a convolutional neural network (CNN). The hardware processor executes the software code to receive video data including a key video frame in color and a video sequence in gray scale, determine a first estimated colorization for each frame of the video sequence except the key video frame based on a colorization of a previous frame, and determine a second estimated colorization for each frame of the video sequence except the key video frame based on the key video frame in color. For each frame of the video sequence except the key video frame, the software code further blends the first estimated colorization with the second estimated colorization using a color fusion stage of the CNN to produce a colorized video sequence corresponding to the video sequence in gray scale.

Video color propagation

According to one implementation, a video processing system includes a computing platform having a hardware processor and a system memory storing a frame interpolation software code, the frame interpolation software code including a convolutional neural network (CNN) trained using a loss function having an image loss term summed with a phase loss term. The hardware processor executes the frame interpolation software code to receive first and second consecutive video frames including respective first and second images, and to decompose the first and second images to produce respective first and second image decompositions. The hardware processor further executes the frame interpolation software code to use the CNN to determine an intermediate image decomposition corresponding to an interpolated video frame for insertion between the first and second video frames based on the first and second image decompositions, and to synthesize the interpolated video frame based on the intermediate image decomposition.

Video frame interpolation using a convolutional neural network

Systems and methods of propagating edits through a video are presented herein. Decompositions of information defining frame images of a video may be obtained. Individual decompositions may include individual sets of sinusoidal functions separated into individual frequency bands. One or more frequency bands that convey differences between an (unedited) frame image and an edited version of that frame image may be identified. A subset of the one or more frequency bands that convey motion information associated with the frame image and a subsequent frame image in the video may be identified. Using the motion information, phase information of individual sinusoidal functions of a decomposition of the edited frame image, which may be included in the subset of the one or more frequency bands, may be propagated based on the motion information.

Systems and methods for propagating edits through a video

Interpolating frames of a video may provide a technique for one or more of frame rate conversion, temporal upsampling for generating slow motion video, image morphing, virtual view synthesis, and/or other video applications. A system may be configured to interpolated frames of a video by leveraging frequency domain representations of individual frames. The frequency domain representations may be decomposed into set of discrete functions that make up the frequency domain representations. Corresponding functions from sets of functions associated with frames with which an interpolated frame is to be determined may be identified. Phase differences between corresponding functions may be determined. Interpolated functions between the corresponding functions may be determined based on the determined phased differences. Information describing spatial domain representations of interpolated frames may be determined based on the interpolated functions.

Systems and methods for interpolating frames of a video

Growing community of inventors

Zurich, Switzerland

Simone Meyer