Vmware, Inc.

Facebook Technologies, LLC

Xingze He

Hari Sivaraman

Uday Pundalik Kurkure

Peter Farkas

Banit Agrawal

Rishi Bidarkar

Samuel Alan Johnson

Elif Albuz

Bryan Michael Anenberg

Nikita Shulga

Colin Lea

In one embodiment, a method includes receiving an audio signal comprising a plurality of speech units, processing the audio signal to associate each of the speech units with a corresponding lip animation, determining pitch information associated with each of the plurality of speech units, processing the pitch information of each of the plurality of speech units to associate at least one of the speech units with a facial-component animation, and presenting the audio signal with a displayed animation of a face, wherein the animation of the face displays the lip animation associated with each of the speech units and the facial-component animation associated with the at least one speech unit. The animation of the face may be displayed in real time with the audio signal. The facial component animation may include animation of the lips, eyebrows, eyelids, and other portion of the upper face.

Audio-based face tracking and lip syncing for natural facial animation and lip movement

Techniques are described for improving the measurement of visual perception of graphical user interface (GUI) information remoted to client devices in virtual desktop environments, such as VDI and DAAS. An objective image quality measurement of remoted virtual desktop interfaces is computed, that is more accurate and more closely aligned with subjective user perception. The visual quality metric is computed using a linear fusion model that combines a peak signal to noise ratio (PSNR) score of the distorted image, a structural similarity (SSIM) score of the distorted image and a feature similarity (FSIM) score of the distorted image. Prior to using the model to compute the quantitative visual perception metric, the linear fusion model is trained by using a benchmark test database of reference images (e.g., virtual desktop interface images), distorted versions of those images and subjective human visual perception quality ratings associated with each distorted version.

Quantitative visual perception quality measurement for virtual desktops

Techniques disclosed herein provide an approach using pixel perturbation to transmit information. In one embodiment, a pixel perturbation engine perturbs pixels of an image to encode meta-information therein. In particular, the perturbation engine may perform controlled perturbation of pixel bits which are unlikely to affect visual quality are perturbed, such as perturbing the least significant bits in one or more channels of a RGB or YUV image. Higher order bits may also be perturbed in color channels such as the blue or UV channels that the human eye is less sensitive to. To mitigate loss of information, the information may be encoded using error correcting codes. After the image having meta-information encoded therein is transmitted to a client device, the meta-information may be recovered and used to determine performance of the virtual desktop infrastructures (VDI) system.

Pixel perturbation for transmission of meta-information

Techniques disclosed herein provide an approach using pixel perturbation to measure image quality. In one embodiment, a pixel perturbation engine perturbs pixels in an image with a reference image for measuring quality of the image after transmission. By perturbing least significant bits, a composite image may be generated in which the reference image is hidden in the original image. The perturbations in the composite image may then be recovered after the composite image is transmitted to a remote device and used to determine image quality based on preservation of the perturbations. In another embodiment, image(s) perturbed with reference image(s) at increasingly higher order bit positions may be transmitted, and quality of the transmitted reference image determined, until the determined quality exceeds a threshold, with the perturbed bit position at which the determined quality exceeds the threshold being indicative of the quality of the image(s) transmitted.

Pixel perturbation for image quality measurement

Growing community of inventors

San Jose, CA, United States of America

Xingze He