Adobe, Inc.

Microsoft Technology Licensing, LLC

Princeton University

Wilmot Li

Hijung Shin

Charles Edward Jacobs

David H Salesin

Deepali Aneja

Matthew David Fisher

Vineet Batra

Ankit Phogat

Lubomira A Dontcheva

David P Simons

Li-Yi Wei

Jovan Popović

Holger Winnemoeller

Kevin Wampler

Daniel Max Kaufman

Adam Finkelstein

Nora Willett

Joel Richard Brandt

Vladimir Kim

Dingzeyu Li

Kim Pascal Pimmel

Stephen Diverdi

Daniel Sýkora

John Nelson

Ashwani Chandil

Morgan Dixon

Rubaiat Habib Kazi

Jasper O'Leary

Marek Dvoroznák

Nazmus Saquib

Rubaiat Habib

Jiaju Ma

Embodiments of the present disclosure provide, a method, a system, and a computer storage media that provide mechanisms for multimedia effect addition and editing support for text-based video editing tools. The method includes generating a user interface (UI) displaying a transcript of an audio track of a video and receiving, via the UI, input identifying selection of a text segment from the transcript. The method also includes in response to receiving, via the UI, input identifying selection of a particular type of text stylization or layout for application to the text segment. The method further includes identifying a video effect corresponding to the particular type of text stylization or layout, applying the video effect to a video segment corresponding to the text segment, and applying the particular type of text stylization or layout to the text segment to visually represent the video effect in the transcript.

Video editing using transcript text stylization and layout

Embodiments are disclosed for constrained modification of vector geometry. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a selection of a first segment of a vector graphic to be edited, identifying an active region associated with the first segment, wherein the active region includes the first segment and at least one second segment which comprise a geometric primitive, identifying the region of influence including at least one third segment connected to the active region, identifying at least one constraint associated with the active region or the region of influence based at least on the geometric primitive, receiving an edit to the active region, and generating an update for the vector graphic based on the edit and the at least one constraint.

Constrained modification of vector graphics

This disclosure involves mapping body movements to graphical manipulations for real-time human interaction with graphics. Certain aspects involve importing graphical elements and mapping input actions, such as gestures, to output graphical effects, such as moving, resizing, changing opacity, and/or deforming a graphic, by using nodes of a reference skeleton and edges (e.g., links) between the nodes of the reference skeleton and the pins. The mapping is used to trigger and interact with the graphical elements with body position and/or movement.

Interactive body-driven graphics for live video performance

This disclosure generally relates to character animation. More specifically, this disclosure relates to pose selection using data analytics techniques applied to training data, and generating 2D animations of illustrated characters using performance data and the selected poses. An example process or system includes extracting sets of joint positions from a training video including the subject, grouping the plurality of frames into frame groups using the sets of joint positions for each frame, identifying a representative frame for each frame group using the frame groups, clustering the frame groups into clusters using the representative frames, outputting a visualization of the clusters at a user interface, and receiving a selection of a cluster for animation of the subject.

Pose selection and animation of characters using video data and training techniques

This disclosure generally relates to character animation. More specifically, but not by way of limitation, this disclosure relates to pose selection using data analytics techniques applied to training data, and generating 2D animations of illustrated characters using performance data and the selected poses. An example process or system includes obtaining a selection of training poses of the subject and a set of character poses, obtaining a performance video of the subject, wherein the performance video includes a plurality of performance frames that include poses performed by the subject, grouping the plurality of performance frames into groups of performance frames, assigning a selected training pose from the selection of training poses to each group of performance frames using the clusters of training frames, generating a sequence of character poses based on the groups of performance frames and their assigned training poses, outputting the sequence of character poses.

Disclosed systems and methods predict visemes from an audio sequence. In an example, a viseme-generation application accesses a first audio sequence that is mapped to a sequence of visemes. The first audio sequence has a first length and represents phonemes. The application adjusts a second length of a second audio sequence such that the second length equals the first length and represents the phonemes. The application adjusts the sequence of visemes to the second audio sequence such that phonemes in the second audio sequence correspond to the phonemes in the first audio sequence. The application trains a machine-learning model with the second audio sequence and the sequence of visemes. The machine-learning model predicts an additional sequence of visemes based on an additional sequence of audio.

Using machine-learning models to determine movements of a mouth corresponding to live speech

Introduced here are computer programs and associated computer-implemented techniques for finding the correspondence between sets of graphical elements that share a similar structure. In contrast to conventional approaches, this approach can leverage the similar structure to discover how two sets of graphical elements are related to one another without the relationship needing to be explicitly specified. To accomplish this, a graphics editing platform can employ one or more algorithms designed to encode the structure of graphical elements using a directed graph and then compute element-to-element correspondence between different sets of graphical elements that share a similar structure.

Algorithmic approach to finding correspondence between graphical elements

Systems and methods for editing an image based on multiple constraints are described. Embodiments of the systems and methods may identify a change to a vector graphics data structure, generate an update for the vector graphics data structure based on strictly enforcing a handle constraint, a binding constraint, and a continuity constraint, adjust the vector graphics data structure sequentially for each of a plurality of sculpting constraints according to a priority ordering of the sculpting constraints, generate an additional update for the vector graphics data structure based on strictly enforcing the binding constraint and the continuity constraint and approximately enforcing the handle constraint and the sculpting constraints, adjust the vector graphics data structure sequentially for each of a plurality of sculpting constraints, and display the vector graphic based on the adjusted vector graphics data structure.

Shape aware transformation

Techniques of managing design iterations include generating data linking selected snapshot histories with contextual notes within a single presentation environment. A designer may generate a design iteration in a design environment. Once the design iteration is complete, the designer may show a snapshot of the design iteration to a stakeholder. The stakeholder then may provide written contextual notes within the design environment. The computer links the contextual notes to the snapshot and stores the snapshot and contextual notes in a database. When the designer generates a new design iteration from the previous design iteration and the contextual notes, the computer generates a new snapshot and a link to the previous snapshot to form a timeline of snapshots. The designer may then present the snapshots, the timeline, and the contextual notes to the stakeholder as a coherent history of how the design of the mobile app evolved to its present state.

Integrated computing environment for managing and presenting design iterations

This disclosure relates to methods, non-transitory computer readable media, and systems that use style-aware puppets patterned after a source-character-animation sequence to generate a target-character-animation sequence. In particular, the disclosed systems can generate style-aware puppets based on an animation character drawn or otherwise created (e.g., by an artist) for the source-character-animation sequence. The style-aware puppets can include, for instance, a character-deformational model, a skeletal-difference map, and a visual-texture representation of an animation character from a source-character-animation sequence. By using style-aware puppets, the disclosed systems can both preserve and transfer a detailed visual appearance and stylized motion of an animation character from a source-character-animation sequence to a target-character-animation sequence.

Generating target-character-animation sequences based on style-aware puppets patterned after source-character-animation sequences

Disclosed systems and methods predict visemes from an audio sequence. A viseme-generation application accesses a first set of training data that includes a first audio sequence representing a sentence spoken by a first speaker and a sequence of visemes. Each viseme is mapped to a respective audio sample of the first audio sequence. The viseme-generation application creates a second set of training data adjusting a second audio sequence spoken by a second speaker speaking the sentence such that the second and first sequences have the same length and at least one phoneme occurs at the same time stamp in the first sequence and in the second sequence. The viseme-generation application maps the sequence of visemes to the second audio sequence and trains a viseme prediction model to predict a sequence of visemes from an audio sequence.

A user interface for improving document layout on arbitrary devices of different resolutions and size using manifold representations of content. Manifold representations of content are: multiple versions of anything that might appear in a document, from text, to images, to even such things as stylistic conventions. The specific content is selected and formatted dynamically, on the fly, by a layout engine in order to best adapt to a given viewing situation. A user interface for authoring and editing such manifold content is disclosed.

System and method for adaptive document layout via manifold content

A system and method for improving document layout on arbitrary devices of different resolutions and size using manifold representations of content. Manifold representations of content are: multiple versions of anything that might appear in a document, from text, to images, to even such things as stylistic conventions. The specific content is selected and formatted dynamically, on the fly, by a layout engine in order to best adapt to a given viewing situation. A user interface for authoring and editing such manifold content is disclosed.

A user interface for a system and method for improving document layout on arbitrary devices of different resolutions and size using manifold representations of content. Manifold representations of content are: multiple versions of anything that might appear in a document, from text, to images, to even such things as stylistic conventions. The specific content is selected and formatted dynamically, on the fly, by a layout engine in order to best adapt to a given viewing situation.

Growing community of inventors

Seattle, WA, United States of America

Wilmot Li