Adobe, Inc.

Stephen Diverdi

Li-Yi Wei

Kazi Rubaiat Habib

Scott David Cohen

Vineet Batra

Ankit Phogat

Jose Ignacio Echevarria Vallespi

Cuong D Nguyen

Sayan Ghosh

Jingwan Lu

Zhaowen Wang

Paul John Asente

Lubomira A Dontcheva

Joel Richard Brandt

Hijung Shin

Dingzeyu Li

Zhifei Zhang

Kim Pascal Pimmel

Wilmot Li

John Nelson

Deepali Aneja

Talin Chris Wadsworth

Tianyi Wang

Yulun Zhang

Jiahao Li

Roberto Montano Murillo

Valerie Lina Head

Rubaiat Habib

Jiaju Ma

Ana Maria Cardenas Gasca

Methods and systems disclosed herein relate generally to body-driven interactions with three-dimensional (3D) layered graphics. The system includes a video capture module that can receive a video stream. The video stream may depict a subject and a 3D layered image, in which the 3D layered image has an associated viewpoint. The system may also include a video processing module that can identify one or more actions performed by the subject. The video processing module can determine a transform operation to be applied to the viewpoint. The transform operation may include at least one of changing the zoom level of the viewpoint, moving the location of the viewpoint, and changing the direction of the viewpoint. The video processing module may apply the transform operation to the 3D layered image and then render the transformed 3D layered image on the video stream.

Systems and methods for body-driven interactions in three-dimension layered images

A computing system captures a first image, comprising an object in a first position, using a camera. The object has indicators indicating points of interest on the object. The computing system receives first user input linking at least a subset of the indicators and establishing relationships between the points of interest on the object and second user input comprising a graphic element and a mapping between the graphic element and the object. The computing system captures second images, comprising the object in one or more modified positions using, the camera. The computing system tracks the modified positions of the object across the second images using the indicators and the relationships between the points of interest. The computing system generates a virtual graphic based on the one or more modified positions, the graphic element, and the mappings between the graphic element and the object.

Interactive virtual graphics with physical objects

Embodiments of the present disclosure provide, a method, a system, and a computer storage media that provide mechanisms for multimedia effect addition and editing support for text-based video editing tools. The method includes generating a user interface (UI) displaying a transcript of an audio track of a video and receiving, via the UI, input identifying selection of a text segment from the transcript. The method also includes in response to receiving, via the UI, input identifying selection of a particular type of text stylization or layout for application to the text segment. The method further includes identifying a video effect corresponding to the particular type of text stylization or layout, applying the video effect to a video segment corresponding to the text segment, and applying the particular type of text stylization or layout to the text segment to visually represent the video effect in the transcript.

Video editing using transcript text stylization and layout

Techniques for responsive video canvas generation are described to impart three-dimensional effects based on scene geometry to two-dimensional digital objects in a two-dimensional design environment. A responsive video canvas, for instance, is generated from input data including a digital video and scene data. The scene data describes a three-dimensional representation of an environment and includes a plurality of planes. A visual transform is generated and associated with each plane to enable digital objects to interact with the underlying scene geometry. In the responsive video canvas, an edit positioning a two-dimensional digital object with respect to a particular plane of the responsive video canvas is received. A visual transform associated with the particular plane is applied to the digital object and is operable to align the digital object to the depth and orientation of the particular plane. Accordingly, the digital object includes visual features based on the three-dimensional representation.

Responsive video canvas generation

Certain aspects and features of this disclosure relate to virtual 3D pointing and manipulation. For example, video communication is established between a presenter client device and a viewer client device. A presenter video image is captured. A 3D image of a 3D object is rendered on the client devices and a presenter avatar is rendered on at least the viewer client device. The presenter avatar includes at least a portion of the presenter video image. When a positional input is detected at the presenter client device, the system renders, on the viewer client device, an articulated virtual appurtenance associated with the positional input, the 3D image, and the presenter avatar. A virtual interaction between the articulated virtual appurtenance and the 3D image appear to a viewer as naturally positioned for the interaction with respect to the viewer.

Virtual 3D pointing and manipulation

Certain embodiments involve flow-based color transfers from a source graphic to target graphic. For instance, a palette flow is computed that maps colors of a target color palette to colors of the source color palette (e.g., by minimizing an earth-mover distance with respect to the source and target color palettes). In some embodiments, such color palettes are extracted from vector graphics using path and shape data. To modify the target graphic, the target color from the target graphic is mapped, via the palette flow, to a modified target color using color information of the source color palette. A modification to the target graphic is performed (e.g., responsive to a preview function or recoloring command) by recoloring an object in the target color with the modified target color.

Flow-based color transfer from source graphic to target graphic

Systems and methods for texture hallucination with a large upscaling factor are described. Embodiments of the systems and methods may receive an input image and a reference image, extract an upscaled feature map from the input image, match the input image to a portion of the reference image, wherein a resolution of the reference image is higher than a resolution of the input image, concatenate the upscaled feature map with a reference feature map corresponding to the portion of the reference image to produce a concatenated feature map, and generate a reconstructed image based on the concatenated feature map using a machine learning model trained with a texture loss and a degradation loss, wherein the texture loss is based on a high frequency band filter, and the degradation loss is based on a downscaled version of the reconstructed image.

Texture hallucination for large-scale image super-resolution

Techniques for interacting with virtual environments. For example, a virtual reality application outputs a three-dimensional virtual reality scene. The application receives a creation of a slicing volume that is positioned within the three-dimensional virtual space. The slicing volume includes virtual elements of an object within the scene. The application projects the slicing volume onto a two-dimensional view. The application displays the two-dimensional view within the three-dimensional virtual reality scene. The application associates a surface of a physical object with the two-dimensional view. The application receives an interaction with the surface of the physical object, and based on the interaction, selects one or more virtual elements.

Selecting objects within a three-dimensional point cloud environment

Disclosed herein are embodiments of systems and computer-implemented methods for extracting a set of discrete colors from an input image. A playful palette may be automatically generated from the set of discrete colors, where the playful palette contains a gamut limited to a blend of the set of discrete colors. A representation of the playful palette may be displayed on a graphical user interface of an electronic device. In a first method, an optimization may be performed using a bidirectional objective function comparing the color gamut of the input image and rendering of a candidate playful palette. Initial blobs may be generated by clustering. In a second method, color subsampling may be performed from the image, and a self-organizing map (SOM) may be generated. Clustering the SOM colors may be performed, and each pixel of the SOM may be replaced with an average color value to generate a cluster map.

Systems and methods of generating playful palettes from images

Growing community of inventors

Berkeley, CA, United States of America

Stephen Diverdi