Avid Technology Limited

Shailendra Mathur

Robert M Brown, Jr

Michel Eid

Robert N Stabler

Thomas P Nadas

Sylvain Girard

Daniel Beaudry

Raymond H Tice

Mathieu Lamarre

Stephen McNeill

Katherine H Cornog

Michael Chilton Sheasby

Timothy H Claman

Daniel Desbois

Roger P Sacilotto, Jr

Robert Gonsalves

Timir K Sinha

Jr Robert M Brown

Roger P Sacilotto

Ray Tice

A large language model (LLM) is used to broaden a search for supporting material for a media project. The LLM is provided with contextual material from the media project and optional grounding material and generates a list of types of supporting material. The list is provided to a machine-learning-based encoder, which encodes the list items into embedding space vectors. A body of source material, which may multimodal, is also encoded into the embedding space. A search engine is used to locate items of source material having embedded space vectors closest to vectors corresponding to the list of types of supporting material. Located items are provided to a media editing application as suggested supporting material. The body of source material may include live external sources. A small language model fine-tuned with a training data set generated by a LLM may be used instead of the LLM.

Using artificial intelligence and machine learning to recommend supporting material for media projects

A universal media difference engine generates a change list specifying the edits required to create an edited revision of a media composition from a base version. The difference engine determines the format of the media composition, locates and installs a plug-in corresponding to the format, and uses the plug-in to parse the composition and generate the change list. The supported compositional formats include formats native to specific media editing applications, as well as interoperable formats. The difference engine is able to convert rich change lists expressed in native form to canonical change lists that are compatible with multiple editing applications. Timeline, mixer configuration, and scene graph composition types are supported. Content management system storage requirements are reduced by storing a base version and change lists instead of multiple revisions of the composition. A media composition recreation engine recreates an edited revision by applying a change list to a prior version.

Difference engine for media change management

Indexing of media sources available to media composition applications, such as video editing applications and digital audio workstations, is extended to sources of an arbitrary type, including non-file type sources. Examples of such sources include devices using baseband video and audio protocols, network ports and physical devices connected via USB, Thunderbolt, etc. The sources are discovered and indexed even when not previously known to the application with the result that any available bitstream becomes available to the application. Applications access the sources via feeders, which are addressable portals through which media essence is received. The indexing methods involve the instantiation of a hierarchy of plug-in software modules by a format-handling subsystem of the application or by a media indexing service servicing one or more applications.

Context-based media indexing

Information about each of the media assets that contribute to a published media composition is needed to determine attribution and contractual terms for the composition as a whole. The generation of a hierarchical asset genealogy that includes information about each of the assets that contribute to a media composition provides a mechanism for recording the full asset contribution history of a composition. The information may include asset ownership, distribution rights, asset structure, talent, and editors involved in generating the asset. For time-based compositions, it may specify which portions of an asset contribute to the composition. A report generator generates asset reports from the asset genealogy without the need to access or read the original composition, which may be represented by a proprietary compositional data model corresponding to an editing applications used to create the composition.

Asset genealogy tracking in digital editing systems

The described methods, systems, and computer program products enable media composition editors to tag compositional objects with editorial rendition rules that specify how rendering decisions are to be made and how source media assets are to be selected and processed to generate an output rendering of a media composition that adheres to the editor's rendition-related choices. The editorially-specified rendition rules include spatial-type rules, temporal-type rules, and color-type rules, and generally pertain to the spatial, temporal, and color ranges and resolutions to be included within a rendition. When a rendition of a media composition is to be generated, a rendition engine interprets and applies the editorial rendition rules to generate and output the rendition in accordance with rendition parameters supplied to the rendition engine by a rendition profile, and, in some cases, by the media composition and the source media assets.

Automated media publishing

Plug-in software modules for a media processing application are deployed in a service-oriented architecture as microservices within a Web-scale environment. Each plug-in module registered within a distributed service hosting environment runs as an instance within a gasket that mediates the interaction of the plug-in instance with the environment. The gaskets convert media essence that is input to or output from the plug-in instances from media files to buffers and vice versa as required. Media essence representing intermediate results may be streamed as media essence channels on a data bus, enabling microservice-based processing to be applied in a live environment. Gaskets subscribe to specific media essence channels on the data bus, and output processed media to a media essence channel corresponding to the function of the plug-in module. The use of plug-in associated media channels on a data bus supports decoupled media processing by the plug-in modules.

Plug-ins as microservices

A synchronized media project, synchronization SDK, and synchronization service enable multiple users to produce a media composition collaboratively. A media project includes composition assets that reference media assets. A synchronization SDK enables multiple applications to edit the media project independent of an internal organization of the project. A project synchronization service synchronizes copies of media projects located on clients with a server-based versioned master project. User-selectable project synchronization modes determine whether media project assets are stored on clients, on the server, or on both, and when media assets are transferred. Synchronized media projects include composition assets in proprietary formats and in interoperable formats. Interoperable composition assets are rendered from proprietary composition assets on a track-by-track basis.

Method of consolidating, synchronizing, and streaming production content for distributed editing of media compositions

Methods and systems for constructing media output by combining multiple input media channels corresponding to different views of a scene. The construction involves combining channels that have different spatial ranges and/or color ranges so as to produce a constructed output having a range that is a composite of input channel spatial and/or color ranges. The constructed output is generated dynamically from available input channels in response to output requirements specifying a view defined in part by spatial and/or color parameter ranges. Available input channels are represented by a single multi-channel source object comprising a hierarchy of grouped and aligned channels, with individual media essence files at the lowest level. Input channel ranges and relationships among the ranges are specified by metadata associated with each channel. During editing, input channels may be added, removed, or changed and the range and nature of the constructed media output are updated dynamically.

Multi-channel video editing

High dynamic range media sensor output is encoded using a hybrid transfer function. The hybrid transfer function conforms to existing industry standards over standard brightness level ranges, such as between black and white reference points, and uses a different function to encode captured signals having intensities below the black point and above the white point. Precision over the expanded dynamic range of captured image data is optimally preserved using a two-byte per pixel representation. The transfer function varies across the full dynamic range without clamping off below the maximum value of the expanded captured intensity range. One hybrid transfer function conforms to BT.1886 up to the white point, and uses a linearly varying encoding gamma above the white point up to 1300% IRE, producing encoded values in a 2.14 fixed point representation. The hybrid transfer function is continuous at the white point, and may also have a continuous gradient there.

Preserving dynamic range precision in integer image formats

Pixel-based and region-based methods, computer program products, and systems for detecting, flagging, highlighting on a display, and automatically fixing edge violations in stereoscopic images and video. The highlighting and display methods involve signed, clamped subtraction of one image of a stereo image pair from the other image, with the subtraction preferably isolated to a region of interest near the lateral edges. Various embodiments include limiting the detection, flagging, and highlighting of edge violations to objects causing a degree of perceptual discomfort greater than a user-set or preset threshold, or to objects having a certain size and/or proximity and/or degree of cut-off by a lateral edge of the left or right eye images of a stereo image pair. Methods of removing violations include automatic or semi-automatic cropping of the offending object, and depth shifting of the offending object onto the screen plane.

Growing community of inventors

Beaconsfield, Canada

Shailendra Mathur