Facebook, Inc.

Yahoo! Inc.

Qdesign Usa, Inc.

Excalibur Ip, LLC

Other

Knapp Investment Company Limited

Auditude, Inc.

Zetta Research and Development—volomedia Series

Volomedia, Inc.

Sergiy Bilobrov

Murgesh Navar

Andrey Yruski

Tom Carhart

Iouri Y Poutivski

Maksim Khadkevich

Jason A Lesperance

Eran Ambar

Andres Hernandez Schafhauser

Ronny Lempel

Edward Bortnikov

Nadav Golbandi

Richard J Beaton

Ronald H Gerhards

David Jacques Vaisey

Albert Chau

Hongfei Ma

Systems, methods, and non-transitory computer-readable media can store fingerprint data corresponding to one or more content items in an index data structure, the index data structure comprising a set of bins, wherein at least one first bin references a list of fingerprinted frames, and wherein at least one fingerprinted frame in the list corresponds to a first content item. A determination is made that one or more fingerprinted frames corresponding to a second content item are to be added to the index data structure. At least one of the fingerprinted frames corresponding to the second content item are added to the list of fingerprinted frames that is referenced by the first bin.

Systems and methods for storing content

Systems, methods, and non-transitory computer-readable media can generate at least one fingerprint based on a set of frames corresponding to a test content item, generate a set of distorted fingerprints using at least a portion of the fingerprint, and determine one or more reference content items using the set of distorted fingerprints, wherein the test content item is evaluated against at least one reference content item to identify matching content.

Systems and methods for identifying matching content

An audio identification system generates audio fingerprints and indexes associated with the audio fingerprints based on discrete and overlapping frames within a sample of an audio signal. The system applies a time-to-frequency domain transform to a time-sequence of frames, which may be filtered. The audio identification system then applies a time-variant transformation (e.g., a Discrete Cosine Transform) to the transformed frames and generates an audio fingerprint and index by selecting sets of coefficients of the time-variant transformation. The system selects coefficients that are less sensitive to possible noise and/or distortions in the underlying signal, such as low-frequency coefficients. The time-variant transformation provides sufficient sampling among the indexes by incorporating the phase information of the frames into the indexes. The system stores the audio fingerprint and other identifying information by index for efficient retrieval and matching of the retrieved fingerprints.

Indexing based on time-variant transforms of an audio signal's spectrogram

An audio identification system accounts for an audio signal's complexity when generating a test audio fingerprint for identification of the audio signal. In particular, the audio identification system determines a complexity of an audio signal to be fingerprinted. For example, the audio signal's complexity may be determined by performance of an autocorrelation on the audio signal. Based on the determined complexity, the audio identification system determines a length of a sample of the audio signal used to generate a test audio fingerprint. A sample having the length is then obtained and used to generate a test audio fingerprint for the audio signal. The test audio fingerprint may be compared to a set of reference audio fingerprints to identify the audio signal.

Generating audio fingerprints based on audio signal complexity

The present disclosure provides identification and continuous monitoring of content, such as broadcast content, in real time, using a hierarchical, distributed approach to content identification. Identification, synchronization and verification are provided. Identification may be performed by an identification server using a reference database of fingerprints. The same or another computing device may be used perform synchronization, such that at least some portion of each fingerprint generated by client devices that form a verification group are synchronized for verification. Verification used to determine whether a client device continues to receive the same content identified during identification compares at least a portion of each fingerprint generated by each client device of a verification group. If a mismatch occurs, identification may be repeated to identify the new content being received by a client device.

Continuous content identification of broadcast content

An audio identification system determines whether a portion of a sample of an audio signal includes silence and generates a test audio fingerprint for the audio signal based on the presence of silence. In one embodiment, the audio identification system uses a value indicating silence for a portion of the test audio fingerprint corresponding to the portion of the audio signal that includes silence. When comparing the test audio fingerprint to reference audio fingerprints, the portion of the test audio fingerprint including the value indicating the presence of silence is not used. In another embodiment, the audio identification system replaces the portion including silence with additive audio and generates a test audio fingerprint for comparison based on the resulting modified sample.

Managing silence in audio signal identification

An audio identification system generates a probe audio fingerprint of an audio signal and determines amount of pitch shifting in the audio signal based on analysis of correlation between the probe audio fingerprint and a reference audio fingerprint. The audio identification system applies a time-to-frequency domain transform to frames of the audio signal and filters the transformed frames. The audio identification system applies a two-dimensional discrete cosine transform (DCT) to the filtered frames and generates the probe audio fingerprint from a selected number of DCT coefficients. The audio identification system calculates a DCT sign-only correlation between the probe audio fingerprint and the reference audio fingerprint, and the DCT sign-only correlation closely approximates the similarity between the audio characteristics of the probe audio fingerprint and those of the reference audio fingerprint. Based on the correlation analysis, the audio identification system determines the amount of pitch shifting in the audio signal.

Detecting distorted audio signals based on audio fingerprinting

An audio identification system generates a reference audio fingerprint associated with an event. The reference audio fingerprint is generated from samples of an audio signal associated with the event captured by multiple devices. To generate the reference audio fingerprint, fingerprints are generated from each sample, and the generated fingerprints are temporally aligned. Fingerprints associated a temporally overlapping portion of the audio signal are averaged, and the average value is associated with the temporally overlapping portion of the audio signal and included in the reference audio fingerprint. The reference audio fingerprint is stored along with identifying information, such as an event name, an event time, an event date, or other information describing the event associated with the audio signal from which the samples were captured.

Generating a reference audio fingerprint for an audio signal associated with an event

A commercial detection system generates a probe audio fingerprint of an audio signal associated with a media stream on a broadcast channel and determines whether the media stream has commercial content based on analysis of the probe audio fingerprint. The commercial detection system determines whether a same match between the probe audio fingerprint and a reference audio fingerprint is observed across multiple broadcast channels. Responsive to the number of same matches exceeding a predetermined threshold, the commercial detection system determines that the media stream has commercial content. The commercial detection system may also apply a trained feature analysis model to extracted acoustic features of the audio signal. The commercial detection system determines whether the media stream has commercial content based on a confidence score assigned to the probe audio fingerprint. The commercial detection system reduces false positive detection using program guide information of the media stream.

Growing community of inventors

Santa Clara, CA, United States of America

Sergiy Bilobrov