Tata Consultancy Services Limited

Sandika Biswas

Brojeshwar Bhowmick

Sanjana Sinha

Jayantrao Mohite

Dipanjan Das

Srinivasu Pappula

Abhijan Bhattacharyya

Ashis Sau

Madhurima Ganguly

Ruddra Dev Roychoudhury

Sayan Paul

Bhushan Gurmukhdas Jagyasi

Kavya Gupta

This disclosure relates generally to methods and systems for emotion-controllable generalized talking face generation of an arbitrary face image. Most of the conventional techniques for the realistic talking face generation may not be efficient to control the emotion over the face and have limited scope of generalization to an arbitrary unknown target face. The present disclosure proposes a graph convolutional network that uses speech content feature along with an independent emotion input to generate emotion and speech-induced motion on facial geometry-aware landmark representation. The facial geometry-aware landmark representation is further used in by an optical flow-guided texture generation network for producing the texture. A two-branch optical flow-guided texture generation network with motion and texture branches is designed to consider the motion and texture content independently. The optical flow-guided texture generation network then renders emotional talking face animation from a single image of any arbitrary target face.

Methods and systems for emotion-controllable generalized talking face generation

This disclosure relates generally to navigation of a tele-robot in dynamic environment using in-situ intelligence. Tele-robotics is the area of robotics concerned with the control of robots (tele-robots) in a remote environment from a distance. In reality the remote environment where the tele robot navigates may be dynamic in nature with unpredictable movements, making the navigation extremely challenging. The disclosure proposes an in-situ intelligent navigation of a tele-robot in a dynamic environment. The disclosed in-situ intelligence enables the tele-robot to understand the dynamic environment by identification and estimation of future location of objects based on a generating/training a motion model. Further the disclosed techniques also enable communication between a master and the tele-robot (whenever necessary) based on an application layer communication semantic.

Navigation of tele-robot in dynamic environment using in-situ intelligence

Conventional state-of-the-art methods are limited in their ability to generate realistic animation from audio on any unknown faces and cannot be easily generalized to different facial characteristics and voice accents. Further, these methods fail to produce realistic facial animation for subjects which are quite different than that of distribution of facial characteristics network has seen during training. Embodiments of the present disclosure provide systems and methods that generate audio-speech driven animated talking face using a cascaded generative adversarial network (CGAN), wherein a first GAN is used to transfer lip motion from canonical face to person-specific face. A second GAN based texture generator network is conditioned on person-specific landmark to generate high-fidelity face corresponding to the motion. Texture generator GAN is made more flexible using meta learning to adapt to unknown subject's traits and orientation of face during inference. Finally, eye-blinks are induced in the final animation face being generated.

Audio-speech driven animated talking face generation using a cascaded generative adversarial network

Traditionally, forecasting models were developed using pest or disease instances collected through pest or disease surveillance. The present disclosure relates to pest forecasting using historical pesticide usage information thereby obviating need for voluminous and time consuming effort of collecting site specific data. Firstly forecasting models for different pests or diseases of different crops are generated based on historical data on pesticide usage and historical weather data collected for a geo-location under consideration. The model is validated and adapted with the current scenario of pests. Current scenario is captured using image samples sent from the field or farm through participatory sensing platform. The images are then analyzed to extract information like actual pest infestation in the field, severity, if there was infestation and the like. This analyses helps to derive the actual pest infestation instances in the field.

Systems and methods for pest forecasting using historical pesticide usage information

Most of the prior art references that generate animations fail to determine and consider head movement data. The prior art references which consider the head movement data for generating the animations rely on a sample video to generate/determine the head movements data, which, as a result, fail to capture changing head motions throughout course of a speech given by a subject in an actual whole length video. The disclosure herein generally relates to generating facial animations, and, more particularly, to a method and system for generating the facial animations from speech signal of a subject. The system determines the head movement, lip movements, and eyeball movements, of the subject, by processing a speech signal collected as input, and uses the head movement, lip movements, and eyeball movements, to generate an animation.

Method and system for generating face animations from speech signal input

Estimating 3D human pose from monocular images is a challenging problem due to the variety and complexity of human poses and the inherent ambiguity in recovering depth from single view. Recent deep learning based methods show promising results by using supervised learning on 3D pose annotated datasets. However, the lack of large-scale 3D annotated training data makes the 3D pose estimation difficult in-the-wild. Embodiments of the present disclosure provide a method which can effectively predict 3D human poses from only 2D pose in a weakly-supervised manner by using both ground-truth 3D pose and ground-truth 2D pose based on re-projection error minimization as a constraint to predict the 3D joint locations. The method may further utilize additional geometric constraints on reconstructed body parts to regularize the pose in 3D along with minimizing re-projection error to improvise on estimating an accurate 3D pose.

Weakly supervised learning of 3D human poses from 2D poses

Speech-driven facial animation is useful for a variety of applications such as telepresence, chatbots, etc. The necessary attributes of having a realistic face animation are: 1) audiovisual synchronization, (2) identity preservation of the target individual, (3) plausible mouth movements, and (4) presence of natural eye blinks. Existing methods mostly address audio-visual lip synchronization, and synthesis of natural facial gestures for overall video realism. However, existing approaches are not accurate. Present disclosure provides system and method that learn motion of facial landmarks as an intermediate step before generating texture. Person-independent facial landmarks are generated from audio for invariance to different voices, accents, etc. Eye blinks are imposed on facial landmarks and the person-independent landmarks are retargeted to person-specific landmarks to preserve identity related facial structure. Facial texture is then generated from person-specific facial landmarks that helps to preserve identity-related texture.

Identity preserving realistic talking face generation using audio speech of a user

A method and system is provided for agriculture field clustering and ecological forecasting. The present application provides a method and system for agriculture field clustering and ecological forecasting based on the clustered agriculture fields, comprises capturing an absolute ground data representing a plurality of field measurements of the agriculture fields; capturing a plurality of weather conditions of the agriculture fields; generating a feature set comprising of said absolute ground data and weather data of the agriculture fields; adaptively clustering the plurality of agriculture fields based on the feature set to generate a cluster; generating a generic forecasting model for ecological forecasting comprising of common features of the feature set in said cluster; selecting at least one feature out of the feature set for generating a plurality of adaptive forecasting model based for ecological forecasting and recommending control measures to a user.

Growing community of inventors

Kolkata, India

Sandika Biswas