Unity Technologies Sf

Luca Fascione

Kimball Darr Thurston, Iii

Peter M Hillman

Dejan Momcilovic

Kenneth Gimpelson

Joseph W Marks

Keith F Miller

Millie Maier

Jonathan S Swartz

Millicent Lillian Maier

Carter Bart Sullivan

Lena Petrovic

Sébastien Nicolas Speierer

A captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene may be processed. To perform the processing, image data of the live action scene having a live actor and the display wall displaying a first rendering of a precursor image is received. Further, precursor metadata for the precursor image displayed on the display wall and display wall metadata for the display wall is determined. An image matte is accessed, where the image matte indicates a first portion associated with the live actor and a second portion associated with the precursor image on the display wall in the live action scene. Pixel display values for a replacement wall image of higher resolution than the precursor image is determined, and the image data of the captured scene is adjusted using the pixel display values and the image matte.

Method for image processing of image data for high-resolution images on a two-dimensional display wall

A captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene may be processed. To perform the processing, image data of the live action scene having a live actor and the display wall displaying a first rendering of a precursor image is received. Further, precursor metadata for the precursor image displayed on the display wall and display wall metadata for the display wall is determined. An image matte is accessed, where the image matte indicates a first portion associated with the live actor and a second portion associated with the precursor image on the display wall Image quality levels for display wall portions of the display wall in the image data is determined, and pixels associated with the display wall in the image data are adjusted to the image quality levels.

Method for image processing of image data for varying image quality levels on a two-dimensional display wall

A captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene may be processed. To perform the processing, stereoscopic image data of the live action scene is received, and display wall metadata of the precursor image is determined. Further, a first portion of the stereoscopic image data comprising the stage element in the live action scene is determined based on the stereoscopic image data and the display wall metadata. A second portion of the stereoscopic image data comprising the display wall in the live action scene with the display wall displaying the precursor image is also determined. Thereafter, an image matte for the stereoscopic image data is generated based on the first portion and the second portion.

Method for image processing of image data for a two-dimensional display wall with three-dimensional objects

The disclosed system modifies luminance of a display associated with a selective screen. The display provides a camera with an image having resolution higher than the resolution of the display by presenting multiple images while the selective screen enables light from different portions of the multiple images to reach the camera. The resulting luminance of the recorded image is lower than a combination of luminance values of the multiple images. The processor obtains a criterion indicating a property of the input image where image detail is unnecessary. The processor detects a region of the input image satisfying the criterion, and determines a region of the selective screen corresponding to the region of the input image. The processor increases the luminance of the display by disabling the region of the selective screen corresponding to the region of the input image.

Increasing resolution and luminance of a display

A captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene may be processed. To perform the processing, image data of the live action scene having a live actor and the display wall displaying a first rendering of a precursor image is received. Further, precursor metadata for the precursor image displayed on the display wall and display wall metadata for the display wall is determined. An image matte is accessed, where the image matte indicates a first portion associated with the live actor and a second portion associated with the precursor image on the display wall in the live action scene. Pixel display values to add or modify an image effect or a visual effect are determined, and the image data is adjusted using the pixel display values and the image matte.

Method for image processing of image data for image and visual effects on a two-dimensional display wall

Methods and systems are presented for determining a virtual focus model for a camera apparatus, the camera apparatus comprising one or more image capture elements and one or more optics device through which light in an optical path passes from a stage environment to at least one of the one or more image capture elements, the stage environment including virtual scene display for displaying a virtual scene.

Calibrating focus/defocus operations of a virtual display based on camera settings

The processor obtains a third pixel value and a second pixel value of the display. The processor determines a desired pixel value range that exceeds the second pixel value of the display. The processor obtains a threshold between the third pixel value of the display and the second pixel value of the display. The processor obtains a function mapping the desired pixel value range to a range between the threshold and the second pixel value. The processor applies the first function to an input image prior to displaying the input image on the display. The display presents the image. Upon recording the presented image, the processor determines a region within the recorded image having a pixel value between the threshold and the second pixel value. The processor increases dynamic range of the recorded image by applying an inverse of the function to the pixel value of the region.

Increasing dynamic range of a virtual production display

Methods and systems are presented for generating a virtual scene rendering usable in a captured scene based on a camera position of a camera in a stage environment, a mapping of a plurality of subregions of a virtual scene display in the stage environment to corresponding positions in the stage environment, and details of a virtual scene element. The details might include a subregion of the plurality of subregions for the virtual scene element where on the virtual scene display the given virtual scene element would, at least in part, appear, and stage subregion depth values. A blur factor for a corresponding subregion might be determined based at least in part on the stage subregion depth value and the virtual subregion depth value. Rendering the virtual scene might take into account the blur factor for the given virtual scene element.

Computing virtual screen imagery based on a stage environment, camera position, and/or camera settings

The disclosed system and method can increase resolution of a display in postprocessing. The processor can obtain multiple images presented on a display, where the display is configured to present the multiple images at a first frame rate higher than a frame rate needed to form a perception of motion. The processor can obtain a mask corresponding to one or more images among the multiple images, where the mask indicates a portion of the one or more images among the multiple images to include in an output image. The processor can increase resolution of the display in proportion to the number of multiple images presented to the display by combining, based on the mask, the one or more images among the multiple images to obtain the output image.

A processor performing postprocessing obtains an input image containing both bright and dark regions. The processor obtains a threshold between a first pixel value of the virtual production display and a second pixel value of the virtual production display. The processor modifies the region according to predetermined steps producing a pattern unlikely to occur within the input image, where the pattern corresponds to a difference between the original pixel value and the threshold. The processor can replace the region of the input image with the pattern to obtain a modified image. The virtual production display can present the modified image. A processor performing postprocessing detects the pattern within the modified image displayed on the virtual production display. The processor calculates the original pixel value of the region by reversing the predetermined steps. The processor replaces the pattern in the modified image with the original pixel value.

A processor calibrates the camera by presenting an input image on the display to obtain a presented image. The camera, arbitrarily positioned relative to the display, records the presented image. The processor obtains the input image via a channel different from the display. The processor obtains an indication of a display region associated with the display. The processor determines an input image region corresponding to the display region, and a recorded image region corresponding to the display region. The processor obtains a first pixel value associated with the input image region and a second pixel value associated with the recorded image region. The processor determines a mapping between the first pixel value and the second pixel value, where applying the mapping to the second pixel value substantially produces the first pixel value. The processor stores an identifier associated with the recorded image region and the mapping.

A processor obtains an input image containing both bright and dark regions. The processor obtains a threshold between a first pixel value and a second pixel value of the display. Upon detecting a region of the input image having an original pixel value above the threshold, the processor can create a data structure including a location of the region in the input image and an original pixel value of the region. The data structure occupies less memory than the input image. The display presents the input image including the region of the image having the original pixel value above the threshold. The processor sends the data structure to a camera, which records the presented image. The processor performing postprocessing obtains the data structure and the recorded image and increases dynamic range of the recorded image by modifying the recorded image based on the data structure.

Methods and systems are presented for generating a virtual scene rendering of a captured scene based on a relative position of a camera and a virtual scene display in a stage environment, along with real and virtual lens effects. The details might include determining the camera position and virtual display position in the stage environment, and determining a depth value of a virtual scene element displayed on the virtual scene display. A desired focus model can then be determined from focus parameters of the camera, the depth value, and a desired lens effect, and an adjusted focus for the virtual scene element can be determined from the desired focus model. The adjusted focus can then be applied to the camera, the image of the virtual scene element on the virtual scene display, or pixels representing the virtual scene element in a composite image captured by the camera.

Growing community of inventors

Wellington, New Zealand

Luca Fascione