Avid Technology Limited

Euclid Discoveries, LLC

University of New Hampshire

Avio Technology, Inc.

Katherine H Cornog

Randy M Fayan

Dane P Kottke

Garth Allen Dickie

Nigel Lee

Jeyun Lee

Myo Tun

John J Guo

James Robert Hamilton

Peter J Fasciano

Robert A Gonsalves

Michael Sporer

Peter Zawojski

Oliver Morgan

Michael Eric Phillips

Shailendra Mathur

Michael D Laird

Michel Rynderman

Stephen McNeill

David F Hoag

Mitrajit Dutta

Videos may be characterized by objective metrics that quantify video quality. Embodiments are directed to target bitrate prediction methods in which one or more objective metrics may serve as inputs into a model that predicts a mean opinion score (MOS), a measure of perceptual quality, as a function of metric values. The model may be derived by generating training data through conducting subjective tests on a set of video encodings, obtaining MOS data from the subjective tests, and correlating the MOS data with metric measurements on the training data. The MOS predictions may be extended to predict the target (encoding) bitrate that achieves a desired MOS value. The target bitrate prediction methods may be applied to segments of a video. The methods may be made computationally faster by applying temporal subsampling. The methods may also be extended for adaptive bitrate (ABR) applications by applying scaling factors to predicted bitrates at one frame size to determine predicted bitrates at different frame sizes. A dynamic scaling algorithm may be used to determine predicted bitrates at different frame sizes.

Selection of video quality metrics and models to optimize bitrate savings in video encoding applications

Videos may be characterized by objective metrics that quantify video quality. Embodiments are directed to target bitrate prediction methods in which one or more objective metrics may serve as inputs into a model that predicts a mean opinion score (MOS), a measure of perceptual quality, as a function of metric values. The model may be derived by generating training data through conducting subjective tests on a set of video encodings, obtaining MOS data from the subjective tests, and correlating the MOS data with metric measurements on the training data. The MOS predictions may be extended to predict the target (encoding) bitrate that achieves a desired MOS value. The target bitrate prediction methods may be applied to segments of a video. The methods may be made computationally faster by applying temporal subsampling. The methods may also be extended for adaptive bitrate (ABR) applications by applying scaling factors to predicted bitrates at one frame size to determine predicted bitrates at different frame sizes. A dynamic scaling algorithm may be used to determine predicted bitrates at the different frame sizes.

Dynamic scaling for consistent video quality in multi-frame size encoding

Video characterization for smart encoding based on perceptual quality optimization

A set of tools in a media composition system for stereoscopic video provides visualizations of the perceived depth field in video clips, including depth maps, depth histograms, time-based depth histogram ribbons and curves displayed in association with a media timeline, and multi-panel displays including views of clips temporally adjacent to a clip being edited. Temporal changes in perceived depth that may cause viewer discomfort are automatically detected, and when they exceed a predetermined threshold, the editor is alerted. Depth grading tools facilitate matching depths in an outgoing clip to those in an incoming clip. Depth grading can be performed automatically upon detection of excessively large or rapid perceived depth changes.

Methods for detecting, visualizing, and correcting the perceived depth of a multicamera image sequence

The problem of watermarking a sequence of images from a motion picture can be divided into two parts. The first part is embedding watermarks in the sequence of images. The second part is detecting embedded watermarks in a target sequence of images where the target sequence may have resulted from one or more attacks on an original sequence of images in which the watermarks were embedded. Motion pictures are watermarked by embedding information in different ways in different images. In general, the information to be embedded is used to define a plurality of watermark images. Each watermark image is an apparent pattern of noise in both signal and frequency domains, and is different from the other watermark images. Preferably, each watermark image is temporally uncorrelated with the other watermark images. Each watermark image is used to modify a corresponding image from the motion picture. To detect the presence of the information in a target sequence of images from a motion picture, the target images are processed to determine an alignment with a watermark sequence. When an alignment with the watermark sequence is determined, a sliding window detector is applied to detect the watermark and read the information embedded by the watermark. By embedding the same information in different ways in different images, and then detecting this information by combining data from the different images, the sensitivity of the detector is increased.

Watermarking of motion pictures

High quality intraframe-only compression of video can be achieved using rate distortion optimization and without resizing or bit depth modification. The compression process involves transforming portions of the image to generate frequency domain coefficients for each portion. A bit rate for each transformed portion using a plurality of scale factors is determined. Distortion for each portion is estimated according to the plurality of scale factors. A scale factor is selected for each portion to minimize the total distortion in the image to achieve a desired bit rate. A quantization matrix is selected according to the desired bit rate. The frequency domain coefficients for each portion are quantized using the selected plurality of quantizers as scaled by the selected scale factor for the portion. The quantized frequency domain coefficients are encoded using a variable length encoding to provide compressed data for each of the defined portions. The compressed data is output for each of the defined portions to provide a compressed bitstream at the desired bit rate.

Fixed bit rate, intraframe compression and decompression of video

A structured watermark may be embedded in data by applying an irregular mapping of variations defined by the structured watermark to frequency domain values representing the data. In particular, the frequency domain representation of the data comprises an ordered set of frequency domain values. The structured watermark is used to define an ordered set of variations to be applied to the frequency domain values. Each variation is a value defined by the structured watermark. An irregular mapping from positions in the ordered set of variations to positions in the ordered set of frequency domain values is defined. This irregular mapping is one-to-one and invertible. Application of the irregular mapping to the set of variations results in a set of values that may appear to be noise both in the frequency domain and in the signal domain of the data. The signal domain of the data may be n-dimensional, and may be a spatial, temporal or other domain from which data may be converted to the frequency domain. The signal domain of the data may be continuous or discrete. Each frequency domain value is modified by the variation mapped to the position of the frequency domain value by the irregular mapping. A frequency domain value may be modified using an additive or multiplicative operation. Using additive embedding, the modifications to the frequency domain values may be effected in the signal domain without computing the frequency domain values of the data by transforming, to the signal domain, the results of applying the irregular mapping to the set of variations. The watermark may be detected in target data by using the inverse of the irregular mapping on the frequency domain representation of the target data. Because the watermark is structured, it may be perceptible in the target data after the target data is processed by the inverse of the irregular mapping. A similarity metric, such as correlation, also can be used to detect the presence of the structured watermark in the processed target data.

Watermarking

In calculating motion between two images, a single channel image may be generated for each image based on measurement of a desired characteristic of those images. Given a desired characteristic (such as edge strength or edge magnitude) in an image, a function measures the strength of the desired characteristic in a region around a pixel in an image. A range of values can represent the likelihood, or measure of confidence, of the occurrence of the desired characteristic in the region around the pixel. Thus, each pixel in the single channel image has a value from the range of values that is determined according to a function. This function operates on a neighborhood in the input image that corresponds to the pixel in the single channel image, and measures the likelihood of occurrence of, or strength of, the desired characteristic in that neighborhood. Two single channel images generated from two input images are analyzed to provide a motion estimate that indicates how the location of characteristics in the images changes from one image to the next image. If the desired characteristic is an edge magnitude or edge strength, then the motion is effectively estimated using a constant edge constraint.

Analyzing motion of characteristics in images

A retiming function that defines a rampable retiming effect is used to generate new audio and video samples at appropriate output times. In particular, for each output time, a corresponding input time is determined from the output time by using the retiming function. The retiming function may be a speed curve, a position curve that maps output times to input times directly or a mapping defining correspondence times between points in the video data and points in the audio data. An output sample is computed for the output time based on at least the data in the neighborhood of the corresponding input time, using a resampling function for the type of media data. Synchronization is achieved by ensuring that the input times determined to correspond to output times for video samples correspond to the input times determined to correspond to the same output times for audio samples.

Performance retiming effects on synchronized data in an editing system

Two images are analyzed to compute a set of motion vectors that describes motion between the first and second images. A motion vector is computed for each pixel in an image at a time between the first and second images. This set of motion vectors may be defined at any time between the first and second images, such as the midpoint. The motion vectors may be computed using any of several techniques. An example technique is based on the constant brightness constraint, also referred to as optical flow. Each vector is specified at a pixel center in an image defined at the time between the first and second images. The vectors may point to points in the first and second images that are not on pixel centers. The motion vectors are used to warp the first and second images to a point in time of an output image between the first and second images using a factor that represents the time between the first and second image at which the output image occurs. The warped images are then blended using this factor to obtain the output image at the desired point in time between the first and second images. The point in time at which the output image occurs may be different from the time at which the motion vectors are determined. The same motion vectors may be used to determine two or more output images at different times between the first and second images. The images may be warped using a technique in which many small triangles are defined in an image corresponding in time to the point in time between the first and second images at which the motion vectors are determined. A transform for each small triangle from the point in time at which the motion vectors are determined to the desired interpolated image time is determined, e.g., the triangle is warped using the motion vectors associated with its vertices. For each pixel in each triangle in the output image, corresponding points in the first and second images are determined, and the first and second images are spatially sampled at these points. These samples for each pixel are combined to produce a value for that pixel in the output image.

Interpolation of a sequence of images using motion analysis

Visibie artifacts in images created using image processing based on motion vector maps may be reduced by providing one or more mechanisms for correcting the vector map. In general, the set of motion vectors is changed by selecting one or more portions of the image. The vectors corresponding to the selected one or more portions are modified. Various image processing operations, such as motion compensated interpolation, may be performed using the changed set of motion vectors.

Correcting motion vector maps for image processing

The invention provides an editing system for editing and combining media material into a resulting media composition. The media may be film or video based material, and also may include audio material. The video may adhere to either NTSC or PAL timing. In response to editing instructions, an Edit Decision List (EDL) is produced that specifies the material that makes up the edited composition, and the order of presentation of this material in the output composition. The EDL specifies the input clips that are taken from the source material using source timecode and similarly defines the output order again according to a new time code, the record time code. The editing system produces an extended EDL that facilitates the downstream processing (after the editing has been completed) for post production tasks requiring the need to know accurately where edit points occur.

Edit decision list for identifying the pull down phase of a video signal

By providing an index that stores information about how to access each grouping of elements in an interleaved data stream and each element within each grouping, random access to each element in the interleaved data stream can be achieved. The index may store a temporal offset for each grouping for interleaved data streams in which the order of data samples in the interleaved data stream is different from the order in which those data samples are used to present the data stream. The information that allows each element in a grouping to be accessed may include information defining the length of each element, or information describing the relative position of each element in a grouping, such as a map table. Such a map table may have an entry for each data type, wherein each entry includes an indication of information used to access a sample of the data type.

Growing community of inventors

Newburyport, MA, United States of America

Katherine H Cornog