Tel Aviv University

Other

Meta Platforms, Inc.

Meta Platforms Technologies, LLC

Mor Research Applications Ltd.

Facebook Technologies, LLC

Mts Investments Inc.

Lior Wolf

Oran Gafni

Ofir Levy

Assaf Mushinsky

Yaniv Taigman

Daniel Cohen-Or

Yehuda Finkelstein

Ziv Freund

Moshe Guttman

Haim Suchowski

Hyunbin Park

Tali Zvi

Michael Mrejen

Yaniv Nechemia Taigman

Yoni Choukroun

David Vazquez

Noam Mor

Adam Polyak

Achiya Nagler

Uri Arieli

Itzik Malkiel

Yonatan Donner

Oron Ashual

Rona Schniberg

A computing device performs generating a first identity encoding representing a first facial identity of the person based on an image of a person, generating a second identity encoding representing a second facial identity different from the first facial identity of the person based on the first identity encoding, generating a source encoding by using an encoder to process a source image of the person having an expression, generating an intermediate image by using a decoder to process the source encoding and the second identity encoding, the intermediate image including a face having the second facial identity and the expression of the person in the source image, and generating an output image by blending the source image with facial features of the face in the intermediate image.

Altering a facial identity in a video stream

Disclosed herein are systems and method for training neural network based decoders for decoding error correction codes, comprising obtaining a plurality of training samples comprising one or more codewords encoded using an error correction code and transmitted over a transmission channel where the training samples are subject to gradual interference over a plurality of time steps and associate the encoded codeword(s) with an interference level and a parity check syndrome at each of the plurality of time steps, using the training samples to train a neural network based decoder to decode codewords encoded using an error correction code by (1) estimating a multiplicative interference included in the encoded codeword(s) based on reverse diffusion applied to the encoded codeword(s) across the time steps, (2) computing an additive interference included in the encoded codewords based on the multiplicative interference, and (3) recovering the codeword(s) by removing the additive interference.

Decoding of error correction codes based on reverse diffusion

A method of designing a nanostructure, comprises: receiving a far field optical response and material properties; feeding the synthetic far field optical response and material properties to an artificial neural network having at least three hidden layers; and extracting from the artificial neural network a shape of a nanostructure corresponding to the far field optical response.

Method and system for characterizing a nanostructure by machine learning

In one embodiment, a method includes receiving a first image depicting a context including one or more persons having one or more respective poses, receiving a second image depicting a target person having an original pose, where the target person is to be inserted into the context depicted in the first image, generating a target segmentation mask specifying a new pose for the target person in the context of the first image based on the first image, generating a third image depicting the target person having the new pose based on the second image and the target segmentation mask, and generating an output image based on the first image and the third image, the output image depicting the one or more persons having the one or more respective poses and the target person having the new pose.

Context-aware human generation in an image

A video generation system is described that extracts one or more characters or other objects from a video, re-animates the character, and generates a new video in which the extracted characters. The system enables the extracted character(s) to be positioned and controlled within a new background scene different from the original background scene of the source video. In one example, the video generation system comprises a pose prediction neural network having a pose model trained with (i) a set of character pose training images extracted from an input video of the character and (ii) a simulated motion control signal generated from the input video. In operation, the pose prediction neural network generates, in response to a motion control input from a user, a sequence of images representing poses of a character. A frame generation neural network generates output video frames that render the character within a scene.

Controllable video characters with natural motions extracted from real-world videos

A technique and system for counting the number of repetitions of approximately the same action in an input video sequence using 3D convolutional neural networks is disclosed. The proposed system runs online and not on the complete video. It analyzes sequentially blocks of 20 non-consecutive frames. The cycle length within each block is evaluated using a deep network architecture and the information is then integrated over time. A unique property of the disclosed method is that it is shown to successfully train on entirely synthetic data, created by synthesizing moving random patches. It therefore effectively exploits the high generalization capability of deep neural networks. Coupled with a region of interest detection mechanism and a suitable mechanism to identify the time scale of the video, the system is robust enough to handle real world videos collected from youtube and elsewhere, as well as non-video signals such as sensor data revealing repetitious physical movement.

System and method for the detection and counting of repetitions of repetitive activity via a trained network

A neural network system for detecting at least one object in at least one image, the system includes a plurality of object detectors. Each object detector receives respective image information thereto. Each object detector includes a respective neural network. Each the neural network including a plurality of layers. Layers in different object detectors are common layers when the layers receive the same input thereto and produce the same output therefrom. Common layers are computed only once during object detection for all the different object detectors.

Cascaded neural network

Disclosed herein a system, a method and a device for generating a voice model for a user. A device can include an encoder and a decoder to generate a voice model for converting text to an audio output that resembles a voice of the person sending respective text. The encoder can includes a neural network and can receive a plurality of audio samples from a user. The encoder can generate a sequence of values and provide the sequence of values to the decoder. The decoder can establish, using the sequence of values and one or more speaker embeddings of the user, a voice model corresponding to the plurality of audio samples of the user.

Generating a voice model for a user

In one embodiment, a method includes generating a keypoint pose and a dense pose for a first person in a first pose based on a first image comprising the first person in the first pose, generating an input semantic segmentation map corresponding to a second person in a second pose based on a second image comprising the second person in the second pose, generating a target semantic segmentation map corresponding to the second person in the first pose by processing the keypoint pose, the dense pose, and the input segmentation map using a first machine-learning model, generating an encoding vector representing the second person based on the second image, and generating a target image of the second person in the first pose by processing the encoding vector and the target segmentation map using a second machine-learning model.

Motion transfer using machine-learning models

A method of recognizing image content, comprises applying to the image a neural network which comprises an input layer for receiving the image, a plurality of hidden layers for processing the image, and an output layer for generating output pertaining to an estimated image content based on outputs of the hidden layers. The method further comprises applying to an output of at least one of the hidden layers a neural network branch, which is independent of the neural network and which has an output layer for generating output pertaining to an estimated error level of the estimate. A combined output indicative of the estimated image content and the estimated error level is generated.

Method and system for image content recognition

A convolutional neural network system for detecting at least one object in at least one image. The system includes a plurality of object detectors, corresponding to a predetermined image window size in the at least one image. Each object detector is associated with a respective down-sampling ratio with respect to the at least one image. Each object detector includes a respective convolutional neural network and an object classifier coupled with the convolutional neural network. The respective convolutional neural network includes a plurality of convolution layers. The object classifier classifies objects in the image according to the results from the convolutional neural network. Object detectors associated with the same respective down-sampling ratio define at least one group of object detectors. Object detectors in a group of object detectors being associated with common convolution layers.

Cascaded convolutional neural network

A technique and system for counting the number of repetitions of approximately the same action in an input video sequence using 3D convolutional neural networks is disclosed. The proposed system runs online and not on the complete video. It analyzes sequentially blocks of 20 non-consecutive frames. The cycle length within each block is evaluated using a deep network architecture and the information is then integrated over time. A unique property of the disclosed method is that it is shown to successfully train on entirely synthetic data, created by synthesizing moving random patches. It therefore effectively exploits the high generalization capability of deep neural networks. Coupled with a region of interest detection mechanism and a suitable mechanism to identify the time scale of the video, the system is robust enough to handle real world videos collected from YouTube and elsewhere, as well as non-video signals such as sensor data revealing repetitious physical movement.

A method for providing an output image, the method includes: determining an importance value for each input pixels out of multiple input pixels of an input image; applying on each of the multiple input pixels a conversion process that is responsive to the importance value of the input pixel to provide multiple output pixels that form the output image; wherein the input image differs from the output image.

Growing community of inventors

Herzliya, Israel

Lior Wolf