Northeastern University

Shiseido Company, Limited

Samsung Electronics Co., Ltd.

Tvision Insights, Inc.

Bithuman Inc

Yun Fu

Bin Sun

Shuyang Wang

Chang Kyu Choi

Sungjoo Suh

Inderbir Sidhu

Yanfeng Liu

Songyao Jiang

Haiyi Mao

Kai Ho Edgar Cheung

Steve Gu

Zhengming Ding

Jaejoon Han

Jae-Joon Han

Wonjun Hwang

Sumin Lee

Yu Yin

Shuhui Jiang

Aneesh Bhat

Ming Shao

Yue Wu

A method for image transformation including receiving, from an electronic device, an input image having represented therein an object having a predefined region with a selected characteristic, extracting, from the input image, an isolated image corresponding to the predefined region, inputting the isolated image into a baseline generator, trained by an offline unbalanced neural network, that generates a new image that represents a modification to the predefined region in which the selected characteristic is replaced with a baseline characteristic, and generating an output image that reflects a modification of the input image to include a representation of the new image.

System and method for image transformation

A system for quantifying viewer engagement with a video playing on a display includes at least one camera to acquire image data of a viewing area in front of the display. A microphone acquires audio data emitted by a speaker coupled to the display. The system also includes a memory to store processor-executable instructions and a processor. Upon execution of the processor-executable instructions, the processor receives the image data and the audio data and determines an identity of the video displayed on the display based on the audio data. The processor also estimates a first number of people present in the viewing area and a second number of people engaged with the video. The processor further quantifies the viewer engagement of the video based on the first number of people and the second number of people.

Systems and methods for assessing viewer engagement

Embodiments identify joints of a multi-limb body in an image. One such embodiment unifies depth of a plurality of multi-scale feature maps generated from an image of a multi-limb body to create a plurality of feature maps each having a same depth. In turn, for each of the plurality of feature maps having the same depth, an initial indication of one or more joints in the image is generated. The one or more joints are located at an interconnection of a limb to the multi-limb body or at an interconnection of a limb to another limb. To continue, a final indication of the one or more joints in the image is generated using each generated initial indication of the one or more joints.

Light-weight pose estimation network with multi-scale heatmap fusion

A system, neural network, and corresponding method generate 3D landmarks associated with an object in a 2D image. An embodiment is a system comprising a neural network detector configured to produce planar coordinates of landmarks at points of the object in the 2D image and a depth coordinate estimator. The planar coordinates include planar coordinate pairs. The depth coordinate estimator is configured to receive the 2D image and the planar coordinates and to estimate a depth coordinate for each planar coordinate pair of each landmark to generate the 3D landmarks. The system reduces network parameters from MB to KB and has better performance relative to state-of-the-art methods. The system may be configured to apply the 3D landmarks for face alignment, virtual face makeup, face recognition, eye gaze tracking, face synthesis, or other face related application.

System and method for generating image landmarks

Embodiments of the present disclosure may include a method for providing an encounter via a virtual assistant with artificial intelligence, the method including detecting, by one or more processors, an encounter request from a user.

Assistant with artificial intelligence

Embodiments of the present disclosure may include a system providing visual assistants with artificial intelligence, including an artificial intelligence large language model engine (LLM)coupled to a computer system.

System providing visual assistants with artificial intelligence

An apparatus and corresponding method for frontal face synthesis. The apparatus comprises a decoder that synthesizes a high-resolution (HR) frontal-view (FV) image of a face from received features of a low-resolution (LR) non-frontal-view (NFV) image of the face. The HR FV image is of a higher resolution relative to a lower resolution of the LR NFV image. The decoder includes a main path and an auxiliary path. The auxiliary path produces auxiliary-path features from the received features and feeds the auxiliary-path features produced into the main path for synthesizing the HR FV image. The auxiliary-path features represent a HR NFV image of the face at the higher resolution. As such, an HR identity-preserved frontal face can be synthesized from one or many LR faces with various poses and may be used in types of commercial applications, such as video surveillance.

Frontal face synthesis from low-resolution images

Embodiments provide functionality for identifying joints and limbs in images. An embodiment extracts features from an image to generate feature maps and, in turn, processes the feature maps using a single convolutional neural network trained based on a target model that includes joints and limbs. The processing generates both a directionless joint confidence map indicating confidence with which pixels in the image depict one or more joints and a directionless limb confidence map indicating confidence with which the pixels in the image depict one or more limbs between adjacent joints of the one or more joints, wherein adjacency of joints is provided by the target model. To continue, indications of the one or more joints and the one or more limbs in the image are generated using the directionless joint confidence map, the directionless limb confidence map, and the target model. Embodiments can be deployed on mobile and embedded systems.

Multi-person pose estimation using skeleton prediction

The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions. The disclosure also provides systems and methods for virtually generating output effects on an input image having a face, for creating dynamic texturing to a lip region of a facial image, for a virtual eye makeup add-on that may include multiple layers, a makeup recommendation system based on a trained neural network model, a method for providing a virtual makeup tutorial, a method for fast facial detection and landmark tracking which may also reduce lag associated with fast movement and to reduce shaking from lack of movement, a method of adjusting brightness and of calibrating a color and a method for advanced landmark location and feature detection using a Gaussian mixture model.

Systems and methods for virtual facial makeup removal and simulation, fast facial detection and landmark tracking, reduction in input video lag and shaking, and a method for recommending makeup

Systems and methods for virtual facial makeup removal and simulation, fast facial detection and landmark tracking, reduction in input video lag and shaking, and method for recommending makeup

Embodiments provide methods and systems for image generation through use of adversarial networks. An embodiment trains an image generator comprising (i) a generator implemented with a first neural network configured to generate a fake image based on a target segmentation, (ii) a discriminator implemented with a second neural network configured to distinguish a real image from a fake image and output a discrimination result as a function thereof and (iii) a segmentor implemented with a third neural network configured to generate a segmentation from the fake image. The training includes (i) operating the generator to output the fake image to the discriminator and the segmentor and (ii) iteratively operating the generator, discriminator, and segmentor during a training period, whereby the discriminator and generator train in an adversarial relationship with each other and the generator and segmentor train in a collaborative relationship with each other.

Segmentation guided image generation with adversarial networks

A system and method are provided to detect, analyze and digitally remove makeup from an image of a face. An autoencoder-based framework is provided to extract attractiveness-aware features to perform an assessment of facial beauty.

Growing community of inventors

Brookline, MA, United States of America

Yun Fu