Adobe, Inc.

Microsoft Technology Licensing, LLC

Nvidia Corporation

The State University of New York

University System of Maryland

Li-Yi Wei

Kazi Rubaiat Habib

Baining Guo

Qi Sun

Heung-Yeung Shum

Kun Zhou

Stephen Diverdi

Stephen J DiVerdi

Xin Tong

Xin Sun

Hijung Shin

Wilmot Wei-Mau Li

Wilmot Li

James P Corbett

Timothy Langlois

Christopher Nuuja

John Simpson

Ryo Suzuki

Brooke Hopper

Jingwan Lu

Paul John Asente

Lubomira A Dontcheva

Xi Wang

Byungmoon Kim

Gavin Stuart Peter Miller

Xu Yang

Ying-Qing Xu

Jose Ignacio Echevarria Vallespi

Arie E Kaufman

Joel Richard Brandt

Joon-Young Lee

Vladimir Kim

Peter-Pike Johannes Sloan

Dingzeyu Li

Stefano Petrangeli

Jonathan Eisenmann

Kim Pascal Pimmel

Deepali Aneja

John Nelson

John Hoon Chung

Tianyi Wang

Wei-Chao Chen

Xinguo Liu

Jin Huang

Xiaohan Shi

Jinwoong Jung

Lvdi Wang

Tenell Glen Rhodes, Jr

Chongyang Ma

Jiahao Li

Rubaiat Habib Kazi

Baoquan Liu

Jianwei Han

Mengqi Peng

Nazmus Saquib

Shaoyu Chen

Peihan Tu

Yilan Chen

Jiaju Ma

Rubaiat Habib

An animation system configured for generating an animation scene that includes at least one animation stylization effect applied to one or more three-dimensional digital objects is described. The animation system includes an interface having a timeline portion and a node graph portion. The timeline portion represents various animation stylization effects as clips arranged chronologically relative to a timeline and the node graph portion includes a node cluster for each clip, where individual node clusters are made up of an animate node, an action node, and an effect node. Input at the timeline portion modifying at least one parameter of the animation scene propagates to the node graph portion, and vice versa. The animation system thus presents dual representations of an animation scene in a manner that enables complex animation customizations while organizing animation effects in a simplified, chronological manner.

Digital object animation authoring interfaces

A computing system captures a first image, comprising an object in a first position, using a camera. The object has indicators indicating points of interest on the object. The computing system receives first user input linking at least a subset of the indicators and establishing relationships between the points of interest on the object and second user input comprising a graphic element and a mapping between the graphic element and the object. The computing system captures second images, comprising the object in one or more modified positions using, the camera. The computing system tracks the modified positions of the object across the second images using the indicators and the relationships between the points of interest. The computing system generates a virtual graphic based on the one or more modified positions, the graphic element, and the mappings between the graphic element and the object.

Interactive virtual graphics with physical objects

Embodiments of the present disclosure provide, a method, a system, and a computer storage media that provide mechanisms for multimedia effect addition and editing support for text-based video editing tools. The method includes generating a user interface (UI) displaying a transcript of an audio track of a video and receiving, via the UI, input identifying selection of a text segment from the transcript. The method also includes in response to receiving, via the UI, input identifying selection of a particular type of text stylization or layout for application to the text segment. The method further includes identifying a video effect corresponding to the particular type of text stylization or layout, applying the video effect to a video segment corresponding to the text segment, and applying the particular type of text stylization or layout to the text segment to visually represent the video effect in the transcript.

Video editing using transcript text stylization and layout

The present disclosure relates to systems, non-transitory computer-readable media, and methods for generating and modifying digital animations based on user interactions with a unique user interface portraying a one-dimensional layer motion element and/or elements for generating and utilizing animation path for digital design objects and animation layers. The disclosed system can provide a dynamic one-dimensional layer motion element that adapts to a selected animation layer and portrays selectable animation frames from the animation layer. The disclosed systems can provide options for generating and modifying various frames of the digital animation based on user interactions with the one-dimensional layer motion element, an animation timeline, and/or a corresponding animation canvas. Additionally, in some embodiments, the disclosed systems also generate path animations with complex animation effects based on user selection of animation paths, digital design objects of animation layers, and corresponding selectable path animation feature tools.

Dynamic path animation of animation layers and digital design objects

The present disclosure relates to method, systems, and non-transitory computer-readable media for generating and configuring a digital stream of video content from a stream-transmitting computing device to include interactive graphical elements that are adjustable by viewer computing devices participating in the digital stream. For example, in one or more embodiments, the disclosed systems receive user input from the stream-transmitting computing device that identifies visual attributes of a graphical element that are adjustable via viewer inputs. In one or more embodiments, the disclosed systems provide the graphical elements within the video content and collect the viewer inputs that adjust the graphical element in accordance with the identified visual attributes. Further, the disclosed systems aggregate and visualize the collected viewer inputs in a graphical presentation.

Aggregating and visualizing viewer interactions with graphical elements within streamed video content

Embodiments are disclosed for generating continuous curve textures based on an input exemplar. A method of generating continuous curve textures may include receiving an input exemplar which represents a repetitive pattern as a plurality of vector curves, generating an input graph representation of the input exemplar which represents a geometry and a topology of the input exemplar, synthesizing an output graph based on the input graph representation, and reconstructing output vector curves from the output graph.

Continuous curve textures

Providing and utilizing a one-dimensional layer motion element to generate and manage digital animations

A method includes accessing a first object in a virtual environment, the first object representing a first asset at a first level of detail (LoD). A second object is generated to represent the first asset at a second LoD having decreased complexity. The method further includes determining a first importance value for the first asset and, based on the first importance value, selecting the first object to represent the first asset. Additionally, the method includes accessing a third object representing the second asset at the first LoD and generating a fourth object representing the second asset at the second LoD. The method further includes determining a second importance value, lower than the first importance value, for the second asset and selecting the fourth object to represent the second asset. The method further includes causing a client device to update a display of the virtual environment by transmitting the selected objects.

Delivering a virtual environment with dynamic level of detail per object

The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize an optimization model for generating vector patterns with complex vector structures. For example, the disclosed systems iteratively optimize the similarity between local input and output neighborhoods that account for clusters. Specifically, based on an input exemplar vector image, the disclosed systems generate a sample input cluster representation for more robust iterative sample optimization and pattern reconstruction. To illustrate, the disclosed systems optimize output cluster configurations based on input clusters such that the output clusters minimize a shape energy and a link energy (e.g., to better preserve shape and structure details from the original vector pattern in the input exemplar vector image). From the output clusters, the disclosed systems can reconstruct additional vector elements to create a new vector image with a synthetic vector pattern.

Utilizing clustering to synthesize vector patterns in digital images

Certain aspects and features of this disclosure relate to virtual 3D pointing and manipulation. For example, video communication is established between a presenter client device and a viewer client device. A presenter video image is captured. A 3D image of a 3D object is rendered on the client devices and a presenter avatar is rendered on at least the viewer client device. The presenter avatar includes at least a portion of the presenter video image. When a positional input is detected at the presenter client device, the system renders, on the viewer client device, an articulated virtual appurtenance associated with the positional input, the 3D image, and the presenter avatar. A virtual interaction between the articulated virtual appurtenance and the 3D image appear to a viewer as naturally positioned for the interaction with respect to the viewer.

Virtual 3D pointing and manipulation

In implementations of systems for augmented reality sketching, a computing device implements a sketch system to generate three-dimensional scene data describing a three-dimensional representation of a physical environment including a physical object. The sketch system displays a digital video in a user interface that depicts the physical environment and the physical object and the sketch system tracks movements of the physical object depicted in the digital video using two-dimensional coordinates of the user interface. These two-dimensional coordinates are projected into the three-dimensional representation of the physical environment. The sketch system receives a user input connecting a portion of a graphical element in the user interface to the physical object depicted in the digital video. The sketch system displays the portion of the graphical element as moving in the user interface corresponding to the movements of the physical object depicted in the digital video.

Systems for augmented reality sketching

This disclosure involves mapping body movements to graphical manipulations for real-time human interaction with graphics. Certain aspects involve importing graphical elements and mapping input actions, such as gestures, to output graphical effects, such as moving, resizing, changing opacity, and/or deforming a graphic, by using nodes of a reference skeleton and edges (e.g., links) between the nodes of the reference skeleton and the pins. The mapping is used to trigger and interact with the graphical elements with body position and/or movement.

Interactive body-driven graphics for live video performance

The present disclosure relates to systems, methods, and non-transitory computer-readable media that generate sample circle representations for digital shapes. For example, the disclosed systems can generate a sample circle representation for a digital shape by selecting a subset of sample circles generated using a medial axis corresponding to the digital shape. In some embodiments, the disclosed systems generate the sample circle representation for the digital shape by generating one or more enclosing circles based on one or more sampled boundary points of the digital shape. In some embodiments, the disclosed systems generate the sample circle representation for the digital shape by further connecting the sample circles generated for the digital shape.

Generating representations to approximate digital shapes using sample circles

Using a prediction engine, generating, based on deformations of prior editing operations performed with a graphics editing tool, suggested editing operations that augment current editing operations applied to a graphical object. The prediction engine accesses first samples defining first positions along first paths of previous editing operations applied to a mesh object in a previous frame and second samples defining second positions along second paths of executed editing operations applied in a current frame. The prediction engine identifies, from a comparison of the first samples and the second samples, a matching component set from the previous editing operations that corresponds to the executed editing operations. The prediction engine deforms the first samples toward the second samples and determines suggested editing operations that comprise a non-matching component set as modified based on the deformed first samples. The prediction engine updates an interface to provide the suggested editing operations.

Generating suggested edits for three-dimensional graphics based on deformations of prior edits

In some embodiments, a computing device uses a blue noise sampling operation to identify source pixels from an input image defining respective pixel sets. Each pixel set is associated with a respective weight matrix for a down-scaling operation. The blue noise sampling operation causes an overlap region between first and second pixel sets. The computing device assigns an overlap pixel in the overlap region to the first weight matrix based on the overlap pixel being closer to the first source pixel. The computing device modifies the second weight matrix to exclude the overlap pixel from a portion of the down-scaling operation involving the second weight matrix. The computing device performs the down-scaling operation on the input image by combining the first pixel set into a first target pixel with the first weight matrix and combining the second pixel set into a second target with the modified second weight matrix.

Image down-scaling with pixel sets selected via blue noise sampling

Growing community of inventors

Redwood City, CA, United States of America

Li-Yi Wei