Ultimatte Corporation

Imatte, Inc.

Other

The Ultimate Corporation Limited

Arpag Dadourian

Paul E Vlahos

Arie Berman

Petro Vlahos

Joseph Parker

George Sauve

A visible mismatch in noise characteristics between a portion of a background scene inserted in composite image by a matte generated from a blue screen and a second portion of the same background inserted by a garbage matte is significantly reduced by adding extracted noise characteristics from the foreground image to the portion of the background scene inserted by the garbage matte. The selective addition of foreground noise characteristics to portions of the background scene significantly enhances the realistic look of a composite image.

Equalization of noise characteristics of the components of a composite image without degrading subject image quality

A white projection screen containing a small number of infrared retro reflective elements, uniformly distributed over the screen's surface, is illuminated by an infrared source and observed by a coaxially located infrared camera. A matte signal is generated utilizing a high signal level from the infrared camera in all unobscured areas on the screen, and a very low signal level in screen areas obscured by a presenter between a projector and the projection screen.

Generating a matte signal from a retro reflective component of a front projection screen

A pattern is projected onto a screen by a projector using an illumination having a non-visible wavelength. A camera, displaced several inches from the projector, observes the projected pattern whose signals are then stored to create a reference frame. The pattern is then projected onto the screen a second time with a presenter present. The camera observes the projected pattern whose signals are then stored to create a current frame, including the presenter, and signals obtained from the reference frame are compared to determine their difference. In unobscured screen areas, the pattern images match. The pattern on the presenter is displaced because of the displaced camera, and no longer matches the reference pattern, thereby identifying the presenter's silhouette area.

Generating an inhibit signal by pattern displacement

Changes in a current image frame from variations in room light level, and automatic camera adjustments, are determined and incorporated into a clear (reference) frame, thereby resulting in a zero difference in the background region when the clear frame and a current frame are subtracted. Any deviation from zero defines the beginning edge of a transition from background to subject. The location of the outer edge of a transition area is further refined by subtracting derivatives of the RGB signals in each frame, and by the use of a fourth color channel when available. The difference of derivitives (d -d ) identifies the subject to background transition area, whose edges define the inner and outer boundary lines. Shadow areas on the backing are retained and may be transferred to the inserted background scene. Small areas on the backing having the same color and luminance as the subject are prevented from printing through the subject by a second &ldquo;derivative&rdquo; alpha channel for the small areas.

Method and apparatus for the automatic generation of subject to background transition area boundary lines and subject shadow retention

An objectionable outlining effect, seen as dark edges and white edges outlining everything in a scene, is eliminated by clipping or limiting the spikes generated by excessive enhancement. Specifically, a method is used for improving the quality of an enhanced video image, while simultaneously maintaining or improving image sharpness, by clipping the RGB levels of the enhanced video image, at the points of enhancement, to upper and lower level bounds representing the signal levels of the video signal prior to its enhancement.

Edge enhancement correction for improved image quality

A video projector shows the desired scene on a projection screen. An infrared source close to the video projector uniformly floods the projection screen with non-visible infrared radiation. An infrared sensitive camera, also close to the video projector, observes the projection screen and sees only the uniform infrared illumination of the screen. Upon entry of a subject into the projected video image, the infrared reflected from the subject will not match that of the projection screen and thus the subject area is identified. All pixels of the projected scene, in the area occupied by the subject, are inhibited before reaching the video projector. The subject may then look directly at an audience without being blinded by the projector.

Method and apparatus for inhibiting projection of selected areas of a projected image

In an image being displayed on a monitor, each observed background color is identified using a cursor to select the dominant colors. Each foreground color is also identified by using the cursor as a selector. For each pixel, several Candidate mattes are computed one for each background color. Each Candidate matte is computed from a single background color and the set of selected foreground colors using any known method. Of the several Candidate mattes, the greater of these Candidates becomes the matte signal for a given pixel. If this matte is 1.0, then the pixel is part of the background. If the matte is zero, then the pixel is part of the subject. If the matte is between 1.0 and 0.0, then the pixel is in a transition area and the contribution of the background to the pixel is that pixel's matte level. A 'Processed Foreground' is generated by removing the contribution of the background from each pixel in the image, resulting in the subject appearing against a black field. The above generated Candidate mattes permit the generation of Processed Foreground Candidates. Of these Processed Foreground Candidates, the smallest is the final Processed Foreground for a given pixel.

Method for removing from an image the background surrounding a selected subject by generating candidate mattes

In most cases, the estimated subject color and/or the estimated background color for a given pixel will have a certain amount of error, which could result in either not enough background removal, or removal of all of the background and some of the foreground. By manually or automatically altering the estimated subject color and/or background color, the errors in matte calculation and subsequent removal of the background, can be minimized or eliminated. Manual alteration of a subject or background color is achieved by using a cursor to select a true color from the background, for example, and inserting it in one or more areas in the background area being partially obscured by the subject. Automatic alteration of one of the estimated subject or background colors involves its computation as a projection in color space of the observed transition pixel color, onto a plane defined by the subject and background reference colors or other defined planes. By treating the shadow as a semitransparent subject, a matte (alpha channel) may be generated to represent the shadow's transition densities, thus permitting the shadow to be regenerated over the new background.

Comprehensive method for removing from an image the background

A computer implemented method to extract a selected subject from its background, by removing the background, including that portion of the background visible through semi transparent areas of the subject, and generating a matte signal containing a record of background levels outside of and within semitransparent subject areas. The observed RGB signal levels of a pixel in the semitransparent transition between a subject and its background, are a mixture of color contributed by the subject, and by the background. The estimated subject color, and the estimated background color, and the observed color of a transition pixel (pixRGB), may be shown as three points in a three dimensional color space.

Method for removing from an image the background surrounding a selected

A method of removing the foreground subject and its shadow from a series of image frames including a subject before a colored backing being photographed by a moving camera during a live broadcast. Such removal results in a series of image frames of the colored backing without the subject, each image frame corresponding to an image frame containing the subject. The method steps employed identify, for each frame, those pixels in the frame occupied by elements of the subject. Those pixels are set to an RGB level of zero, to create an image of the colored backing in which the subject area is reduced to black. The image of the colored backing is then smeared to obliterate the defined edges of the subject's shadow and to smear the color of the backing into the black area previously occupied by the subject to generate a clear frame. The clear frame is then utilized to compensate for backing luminance and color nonuniformity, thereby preventing visibility of the joining edge of generated windows that extend the backing to the edge of the frame.

Backing luminance non-uniformity compensation in real-time compositing

The invention utilizes the fact that anyone with normal color vision has the ability look at a scene on a video display and recognize a black area if one is present; a white area if one is present, and face or flesh areas if a person is present. It is also easy to recognize blond hair, subjects that are opaque, subjects that are semi-transparent, and subjects that are bright green. The aforesaid abilities to recognize luminance and color characteristics of elements within a video scene are used in conjunction with the invention to produce ideal composite images by utilizing of a set of equations that incorporate a knowledge of color science, of video and existing compositing techniques. These equations are used to make adjustment decisions to produce the best possible composite image. The equations are used to compute certain constant values utilized in the existing compositing techniques. The equations are also used to calculate exact values of B, G, R and gamma for balancing blacks, whites, and flesh tones as well as foreground and background video levels.

Method and apparatus for adjusting parameters used by compositing devices

A method and apparatus which allows different amounts of filtering to be applied to specific parts of a foreground image, thus minimizing visible noise in the backing area, while preserving fine detail information of foreground objects. The method and apparatus identify three areas within the foreground, namely unobscured backing (screen) area, screen to foreground subject transition area, foreground subject area. The area where full amount of filtering is desired is the backing area, including shadows. This is the area where the background image will be added, and if noise is present in this area, it will be added to the background image. The transition area from the screen to the foreground object requires variable amounts of filtering, depending on the width of the transition. For sharply focused edges and very fine detail, the transition width is very small, and any filtering applied there will cause blurring and softening of these edges and details. Therefore, no filtering is applied to those areas. For out of focus and moving objects, the transition width is large, and no foreground detail information is visible. In these areas, a variable amount of filtering is applied, based on the width of the transition. In this manner, noise in these transition areas is significantly reduced, without much noticeable change in already blurred edges. Within foreground subject areas, no filtering is applied, thus preserving the detail of the foreground subject.

Screen filtering boundary detection for noise filtering prior to image

A method and apparatus to improve the control signal E.sub.c used in a video image compositing system so that the adjustment to achieve an E.sub.c of just zero for green objects and flesh tones, does not simultaneously reduce E.sub.c in grey scale subjects and cause print-through. The invention also reproduces a wide range of blue colors from pale blue to bright blue in the presence of a blue backing without raising a matte density control and without raising noise level. The invention also provides a second control signal, E.sub.k, that eliminates backing color spill from blond and brown hair, without affecting bright foreground white tones. The invention also generates a SHADOW CLEAN-UP signal that eliminates foreground shadows, while preserving most of the foreground detail information.

Growing community of inventors

Northridge, CA, United States of America

Arpag Dadourian