Adobe, Inc.

Amol Jindal

Ajay Bedi

Anmol Dhawan

Subham Gupta

Poonam Bhalla

Krishna Singh Karki

Mahika Wason

Varun Maini

Srinivasa Madhava Phaneen Angara

Vivek Mishra

Neha Sharan

The present disclosure describes systems, non-transitory computer-readable media, and methods that can generate contextual identifiers indicating context for frames of a video and utilize those contextual identifiers to generate translations of text corresponding to such video frames. By analyzing a digital video file, the disclosed systems can identify video frames corresponding to a scene and a term sequence corresponding to a subset of the video frames. Based on images features of the video frames corresponding to the scene, the disclosed systems can utilize a contextual neural network to generate a contextual identifier (e.g. a contextual tag) indicating context for the video frames. Based on the contextual identifier, the disclosed systems can subsequently apply a translation neural network to generate a translation of the term sequence from a source language to a target language. In some cases, the translation neural network also generates affinity scores for the translation.

Translating texts for videos based on video context

A system identifies a video comprising frames associated with content tags. The system detects features for each frame of the video. The system identifies, based on the detected features, scenes of the video. The system determines, for each frame for each scene, a frame score that indicates a number of content tags that match the other frames within the scene. The system selects, for each scene, a set of key frames that represent the scene based on the determined frame scores. The system receives a search query comprising a keyword. The system generates, for display, search results responsive to the search query including a dynamic preview of the video. The dynamic preview comprises an arrangement of frames of the video corresponding to each scene of the video. Each of the arrangement of frames is selected from the selected set of key frames representing the respective scene of the video.

Adaptive search results for multimedia search queries

Techniques are provided for identifying objects (such as products within a physical store) within a captured video scene and indicating which of object in the captured scene matches a desired object requested by a user. The matching object is then displayed in an accentuated manner to the user in real-time (via augmented reality). Object identification is carried out via a multimodal methodology. Objects within the captured video scene are identified using a neural network trained to identify different types of objects. The identified objects can then be compared against a database of pre-stored images of the desired product to determine if a close match is found. Additionally, text on the identified objects is analyzed and compared to the text of the desired object. Based on either or both identification methods, the desired object is indicated to the user on their display, via an augmented reality graphic.

Image processing techniques to quickly find a desired object among other objects from a captured video scene

Techniques are described for trace layer for replicating a source region of a digital image. In an implementation, a user leverages a content editing system to select a source region of a source image to be replicated and a target region of a target image to which portions of the source region are to be replicated. A trace layer is generated that is a visual representation of portions of the source region, and the trace layer is positioned on the target region of the target image. Further, the trace layer is generated based on a visibility factor such that the trace layer is at least partially transparent. The trace layer receives user interaction to select portions of the trace layer and visibility of the selected portions is modified to replicate corresponding portions of the source region to the target region.

Trace layer for replicating a source region of a digital image

Certain embodiments involve adaptive search results for multimedia search queries to provide dynamic previews. For instance, a computing system receives a search query that includes a keyword. The computing system identifies, based on the search query, a video file having keyframes with content tags that match the search query. The computing system determines matching scores for respective keyframes of the identified video file. The computing system generates a dynamic preview from at least two keyframes having the highest matching scores.

The present disclosure relates to systems, methods, and non-transitory computer-readable media that utilize a multi-panel graphical user interface for modifying digital images. For example, in one or more embodiments, the disclosed systems divide the graphical user interface of a client device into first panel and a second panel. Further, the disclosed systems provide different portions of a digital image for display within the first and second panels. In some implementations, the disclosed systems receive a user interaction with the portion of the digital image displayed within the first panel. Based on the received user interaction, the disclosed systems modify the second portion of the digital image displayed within the second panel.

Editing digital images using multi-panel graphical user interfaces

This disclosure relates to methods, non-transitory computer readable media, and systems that can generate a context-aware-video-progress bar including a video-scene-proportionate timeline with time-interval sections sized according to relative scene proportions within time intervals of a video. In some implementations, for instance, the disclosed systems determine relative proportions of scenes within a video across time intervals of the video and generate a video-scene-proportionate timeline comprising time-interval sections sized proportionate to the relative proportions of scenes across the time intervals. By integrating the video-scene-proportionate timeline within a video-progress bar, the disclosed systems generate a context-aware-video-progress bar for a video. Such a context-aware-video-progress bar can facilitate more precise and intelligent scrubbing through a video, a dynamic graphical user interface for navigating within and identifying frames of the video, and a flexible user-friendly tool for quickly identifying scenes.

Providing context aware video searching

A seek content extraction system analyzes frames of video content and identifies locations in the frames where session information is displayed. This session information refers to information that is displayed as part of video content and that describes, for a particular location in the video content, what is currently happening in the video content at that particular location. This session information is extracted from each of multiple frames, and for a given frame the extracted session information is associated with the frame. While the user is seeking forward or backward through the video content, a thumbnail of the frame at a given location in the video content is displayed along with the extracted session information associated with the frame.

Extracting session information from video content to facilitate seeking

The present disclosure relates to systems, methods, and non-transitory computer readable media for generating and providing composition effect tutorials for creating and editing digital content based on a metadata composite structure. For example, the disclosed systems can generate and/or access a metadata composite structure that includes nodes corresponding to composition effects applied to a digital content item, where a given node can include location information indicating where a composition effect is applied relative to a digital content item. The disclosed systems can further generate a tutorial to guide a user to implement a selected composition effect by identifying composition effects of nodes that correspond to a location selected within a composition interface and presenting instructions for a particular composition effect.

Generating and providing composition effect tutorials for creating and editing digital content

Salient video frame establishment is described. In one or more example embodiments, salient frames of a video are established based on multiple photos. An image processing module is capable of analyzing both video frames and photos, both of which may include entities, such as faces or objects. Frames of a video are decoded and analyzed in terms of attributes of the video. Attributes include, for example, scene boundaries, facial expressions, brightness levels, and focus levels. From the video frames, the image processing module determines candidate frames based on the attributes. The image processing module analyzes multiple photos to ascertain multiple relevant entities based on the presence of entities in the multiple photos. Relevancy of an entity can depend, for instance, on a number of occurrences. The image processing module establishes multiple salient frames from the candidate frames based on the multiple relevant entities. Salient frames can be displayed.

Growing community of inventors

Patiala, India

Amol Jindal