Personify, Inc.

University of Illinois

Pesonify, Inc.

Dennis Lin

Quang Nguyen

Sanjay J Patel

Minh Do

Simion Venshtain

Sumant Kowshik

Cong Nguyen

Long Dang

Daniel P Dabbelt

Glenn Francisco

Yi Zhang

Gia Dang

Mert Dikmen

Long Truong

James J Kosmach

Quang H Nguyen

Daniel P Dabbeit

David Scherba

Greg Meyer

John Nicholas Timmons

Fernando Rossi Goncalves

Quang Il Nguyen

A system for background image subtraction includes a computing device coupled with a 3D video camera, a processor of the device programmed to receive a video feed from the camera containing images of one or more subject that include depth information. The processor, for an image: segments pixels and corresponding depth information to identify foreground regions associated with a target user.

System for background subtraction with 3D camera

Embodiments disclose systems and methods for transmitting user-extracted video and content more efficiently by recognizing that user-extracted video provides the potential to treat parts of a single frame of a user-extracted video differently. An alpha mask of the image part of the user-extracted video is used when encoding the image part so that it retains a higher quality upon transmission than the remainder of the user-extracted video.

Transmitting video and sharing content via a network

Disclosed herein are methods and systems for classifying pixels as foreground using both short-range depth data and long-range depth data. One embodiment takes the form of a process that includes obtaining video data depicting at least a portion of a user. The process also includes obtaining short-range depth data associated with the video data. The process also includes obtaining long-range depth data associated with the video data. The video data, short-range depth data, and long-range depth data may be obtained via a single 3-D video camera. The process also includes classifying pixels of the video data as foreground based at least in part on both the short-range depth data and the long-range depth data. In some embodiments, classifying pixels of the video data as foreground comprises employing an alpha mask. The alpha mask may comprise binary foreground (hard) indicators. The alpha mask may comprise foreground-likelihood (soft) indicators.

Methods and systems for classifying pixels as foreground using both short-range depth data and long-range depth data

Disclosed herein are methods and systems for assigning pixels distance-cost values using a flood fill technique. One embodiment takes the form of a process that includes obtaining video data depicting a head of a user, obtaining depth data associated with the video data, and selecting seed pixels for a flood fill at least in part by using the depth information. The process also includes performing the flood fill from the selected seed pixels. The flood fill assigns respective distance-cost values to pixels of the video data based on position-space cost values and color-space cost values. In some embodiments, the process also includes classifying pixels of the video data as foreground based at least in part on the assigned distance-cost values. In some other embodiments, the process also includes assigning pixels of the video data foreground-likelihood values based at least in part on the assigned distance-cost values.

Methods and systems for assigning pixels distance-cost values using a flood fill technique

Disclosed herein are systems and methods for iterative adjustment of video-capture settings based on identified persona. In an embodiment, a method includes receiving video frames that are being captured by a video camera of an ongoing scene. The method also includes identifying a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels classified as being representative of the persona. The method also includes selecting, based collectively on the brightness values of the pixels in the identified set, a respective adjustment command for one or more adjustable video-capture settings of the camera, as well as outputting the selected commands to the camera for use in continuing to capture video data representative of the ongoing scene.

Systems and methods for iterative adjustment of video-capture settings based on identified persona

Systems and methods for using geometric primitives to identify background in video data. In one embodiment, a method obtains video data depicting at least a portion of a user. The video data is processed with at least one persona identification module comprising a geometric primitive module for generating a first persona probability map at least in part by: detecting at least one geometric primitive within the video data; identifying a respective region within the video data associated with each of the at least one detected geometric primitives; and assigning the respective regions an increased background-probability in the first persona probability map; and outputting a persona image by extracting pixels from the video data based on the persona probability map.

Methods and systems for identifying background in video data using geometric primitives

A system for background image subtraction includes a computing device coupled with a 3D video camera, a processor o&#xa3; the device programmed to receive a video feed from the camera containing images of one or more subject that include depth information. The processor, for an image: segments pixels and corresponding depth information into three different regions including foreground (FG), background (BG), and unclear (UC); categorizes UC pixels as FG or BG using a function that considers the color and background history (BGH) information associated with the UC pixels and the color and BGH information associated with pixels near the UC pixels; examines the pixels marked as FG and applies temporal and spatial filters to smooth boundaries of the FG regions; constructs a new image by overlaying the FG regions on top of a new background; displays a video feed of the new image in a display device; and continually maintains the BGH.

Embodiments disclose extracting a user persona from a video of arbitrary duration and associating that persona with text for a chat session. Embodiments cause the persona to be extracted at the moment text is sent or received to convey the body language associated with the text.

Integrating user personas with chat sessions

Disclosed herein are systems and methods for persona identification using combined probability maps. An embodiment takes the form of a method that includes obtaining at least one frame of pixel data; processing the at least one frame of pixel data to generate a hair-identification probability map; and generating a persona image by extracting pixels from the at least one frame of pixel data based at least in part on the generated hair-identification probability map.

Systems and methods for persona identification using combined probability maps

Described herein are methods and systems for visually deemphasizing a displayed persona. At least one embodiment takes the form of a method carried out by a user-interface (UI)-management device. The UI-management device identifies a persona region of a displayed persona and identifies a user-interface-focus location. The UI-management device then makes a persona-deemphasize determination based on the identified persona region and the identified user-interface-focus location. In response to making the persona-deemphasize determination, the UI-management device visually deemphasizes the displayed persona.

Methods and systems for visually deemphasizing a displayed persona

A color image and a depth image of a live video are received. Each of the color image and the depth image are processed to identify the foreground and the background of the live video. The background of the live video is removed in order to create a foreground video that comprises the foreground of the live video. A control input may be received to control the embedding of the foreground video into a second background from a background feed. The background feed may also comprise virtual objects such that the foreground video may interact with the virtual objects.

Systems and methods for embedding a foreground video into a background feed based on a control input

Disclosed herein are methods and systems for generating a user-hair-color model. One embodiment takes the form of a process that includes obtaining video data depicting a head of a user. The process also includes determining a set of line segments of pixels of the video data, wherein each line segment in the determined set of line segments intersects an upper contour of the depicted head of the user. The process also includes grouping at least some of the pixels of at least one of the line segments in the determined set of line segments into three sets of pixels based at least in part on respective color data of the pixels. The three sets of pixels include a skin-pixel set, a hair-pixel set, and a background-pixel set. The process also includes updating a user hair-color model based at least in part on the skin-pixel set.

Methods and systems for generating a user-hair-color model

A system for background image subtraction includes a computing device coupled with a 3D video camera, a processor of the device programmed to receive a video feed from the camera containing images of one or more subject that include depth information. The processor, for an image: segments pixels and corresponding depth information into three different regions including foreground (FG), background (BG), and unclear (UC); categorizes UC pixels as FG or BG using a function that considers the color and background history (BGH) information associated with the UC pixels and the color and BGH information associated with pixels near the UC pixels; examines the pixels marked as FG and applies temporal and spatial filters to smooth boundaries of the FG regions; constructs a new image by overlaying the FG regions on top of a new background; displays a video feed of the new image in a display device; and continually maintains the BGH.

Disclosed herein are systems and methods for iterative adjustment of video-capture settings based on identified persona. In an embodiment, a method includes receiving video frames being captured by a video camera of an ongoing scene. The method also includes identifying a persona in one or more of the received frames at least in part by identifying, in each such frame, a set of pixels that is representative of the persona in the frame and that does not include any pixels representative of a background of the frame. The method also includes selecting, based collectively on the brightness values of the pixels in the identified set of pixels of one or more frames, an adjustment command for one or more adjustable video-capture settings of the camera, as well as outputting the selected commands to the camera for use in continuing to capture video data representative of the ongoing scene.

Growing community of inventors

Chicago, IL, United States of America

Dennis Lin