Adobe, Inc.

Nico Becherer

Sven Duwenhorst

Oliver Wang

William L Marino

Nicolas Huynh Thien

Allison Breanne Walke

Jonathan Worthen Brandt

Shabnam Ghadar

Federico Perazzi

Markus Woodson

Alvin Ghouas

Durin Gleaves

Hartmut Warncke

Ben Bowles

Alexander Riss

Nikhil Kalra

Dhaval Shah

Falk Ferdinand Albrecht Oswald

Tim Converse

Sudharshan Reddy Kakumanu

The present disclosure describes systems and methods for a privacy sensitive computing system. One or more embodiments provide a protected computing environment, a code authorization unit, and a data aggregation unit. For example, some embodiments of the privacy sensitive computing system may train unsupervised or self-supervised ML models on user-generated assets subject to privacy considerations that mandate those assets are not viewed directly by human eyes.

Platform for unsupervised machine learning training on unseeable user generated assets

Embodiments of the present invention are directed towards reframing videos from one aspect ratio to another aspect ratio while maintaining visibility of regions of interest. A set of regions of interest are determined in frames in a video with a first aspect ratio. The set of regions of interest can be used to estimate an initial camera path. An optimal camera path is determined by leveraging the identified regions of interest using the initial camera path. Sub crops with a second aspect ratio different from the first aspect ratio of the video are identified. The sub crops are placed as designated using the optimal camera path to generate a cropped video with the second aspect ratio.

Intelligent video reframing

A computer-implemented method for audio signal processing includes analyzing a foreground audio signal to determine metrics corresponding to audio slices of the foreground audio signal. Each such metric indicates a value for an audio property of a respective audio slice. The method further includes computing a total metric for an audio slice as a function of a set of the metrics corresponding to a set of the audio slices including the audio slice. The method further includes adding a key frame to a track based on the total metric. The track includes the foreground audio signal and a background audio signal, and a location of the key frame corresponds to a location of the audio slice on the track. The key frame indicates a change to the audio property of the background audio signal at the location on the track, and the key frame is utilizable for audio ducking.

Automatic audio ducking with real time feedback based on fast integration of signal levels

In implementations of determining video cuts in video clips, a video cut detection system can receive a video clip that includes a sequence of digital video frames that depict one or more scenes. The video cut detection system can determine scene characteristics for the digital video frames. The video cut detection system can determine, from the scene characteristics, a probability of a video cut between two adjacent digital video frames having a boundary between the two adjacent digital video frames that is centered in the sequence of digital video frames. The video cut detection system can then compare the probability of the video cut to a cut threshold to determine whether the video cut exists between the two adjacent digital video frames.

Determining video cuts in video clips

Various embodiments describe audio signal processing. In an example, a computer system generates metrics, such as RMS levels, for audio slices from a foreground audio signal. A summed-area table is generated from the metrics. An observation window is used to determine whether to add a key frame or not. The observation window includes a set of audio slices. A total metrics, such as an average RMS level, is computed for the audio slices in the observation window. Based on the total metric, the computer system adds a key frame. The key frame references audio ducking parameters applicable to a background audio signal.

Average pixel luminosity is calculated for each frame comprising a content item. For each pair of adjacent frames, an IFD is calculated. The IFD represents the difference between a baseline pixel luminosity associated with each of the two frames. An initial set of cut frames is selected based on IFD values that are less than a minimum value IFD, or that are greater than a maximum value IFD. The positions of these initial cut frames are optimized using a numerical optimization technique that favors removal of frames corresponding to IFD extrema, but that also attempts to maintain a minimum time gap between cut frames. Selecting frames for removal is approached as a constraint minimization problem. Once an optimized set of cut frames is established, audio is cut and crossfaded in a temporal window surrounding cut frame positions.

Multimedia content duration manipulation

A digital medium environment includes an audio processing application that generates a user interface for editing audio data. Sound effects are displayed each with a single adjuster according to an ordered workflow based on a mix type of an audio clip, such as music, dialogue, etc. Adjustment of a single adjuster for a sound effect causes adjustment of multiple parameters of an effects processor underlying the sound effect. Furthermore, effects processors corresponding to selected sound effects are placed in an effects rack in a correct order, rather than in an order in which the sound effects are selected, thus correcting user mistakes. The techniques described herein allow users of varied skill levels to work on a same project, and provide a dynamic interface suitable to the user's skill level that also facilitates comparative testing with an audio effect disabled and subsequently enabled without dropouts in the rendered audio.

Digital audio data user interface customization based on user expertise, content type, or testing

A digital medium environment includes an audio editing application that remixes an original audio recording to play back over a target duration. An improved audio remixing method implemented by the audio editing application comprises segmenting the original audio recording into individual beats, computing costs for transitioning between different beats of the original audio recording, and creating a remixed audio recording using the transition costs and beats of the original audio recording. As the remixed audio recording is created, transition costs are updated by assigning penalties to transitions as the transitions are inserted into the remix. Using the beats of the original audio recording with updated transition costs ensures that a remix retains a quality and structure of the original audio recording while avoiding repetition. In some implementations, the audio editing application performs remixing to extend playback of an original audio recording to a longer duration while preserving a beat-level structure of the original audio recording.

Automatic audio remixing with repetition avoidance

Methods and apparatus are presented for a morphing search tool that provides a single user interface through which a user may both modify a displayed image and perform an image similarity query based on the modified image. The morphing search tool may allow a user to repeat this process until a desired image is found and displayed. Further, the image repository to be searched may be specified by a user and the images to be modified and searched for may be any type of image. The morphing search tool may be integrated within a merchant website, an image management system, or as a stand alone application.

Single user interface for selecting, modifying, and querying images

Techniques for utilizing sets of filters to reduce a large number of searchable assets to a meaningful or reduced number of searchable assets. Feature information may be extracted from a particular asset of a set of training assets to create an artificial term. A reduction ratio may then be calculated by utilizing the created artificial term to filter the set of training assets. The reduction ratio may represent the ratio of training assets that contain a particular artificial term. A plurality of filters and their associated reduction ratios may be created this way by utilizing the set of training assets. This process can also involve receiving a requested reduction ratio associated with a set of searchable assets. A combination of filters may then be selected which, when applied to the set of searchable assets, results in a reduced number of searchable assets according to the requested reduction ratio.

Filter selection in search environments

Each of multiple images is analyzed to determine how the colors of the pixels of the image are distributed throughout the color space of the image. Different covariance based characteristics of the image are determined that identify a direction, as well as magnitude in each direction, of the distribution of colors of the image pixels. These different covariance based characteristics that are determined for an image can be saved as associated with the image, allowing the characteristics to be accessed and used as a basis for searching the images to identify particular types of images. These different covariance based characteristics can also be used to order the images identified by a search.

Growing community of inventors

Hamburg, Germany

Nico Becherer