Google Inc.

Gheorghe Postelnicu

Matthew Nirvan Sharifi

Dominik Roblek

Yaniv Bernstein

Sergey Ioffe

Annie Chen

George Tzanetakis

Matthew Wiseman

Richard Tucker

Michele Mae Covell

Jay Yagnik

Jay Michael Ponte

Daniel Jules Switkin

Aviv Reznik

Daniel M Keysers

David Marwood

David A Ross

Varun Kacholia

Johan Granström

Thomas Chadwick Walters

Oliver Heckmann

Jasmine Kent

Nicola Muscettola

Thomas Bugnon

Matthias Rochus Konrad

Hanna Pasula

Murali Krishna Viswanathan

Sai Suman Cherukuwada

Gertjan Pieter Halkes

Boris Nikolaev Daskalov

Khalid Walid Abdel Monem Ashmawy

Henrik Rydgard

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving audio data encoding an utterance and environmental data, obtaining a transcription of the utterance, identifying an entity using the environmental data, submitting a query to a natural language query processing engine, wherein the query includes at least a portion of the transcription and data that identifies the entity, and obtaining one or more results of the query.

Answering questions using environmental context

Systems and techniques for adding pitch shift resistance to an audio fingerprint are presented. In particular, an audio track for a media file is received. A first audio fingerprint for the audio track with a first pitch shift and an Nth audio fingerprint for the audio track with an Mth pitch shift are generated, where N is an integer greater than or equal to two and M is an integer greater than or equal to two. A combined audio fingerprint is generated from at least the first audio fingerprint and the Nth audio fingerprint.

System and method for adding pitch shift resistance to an audio fingerprint

Aspects relate to determining whether a probe media content matches one or more reference media content. The reference media content is classified into a content class. The probe media content could also be classified into a content class. Similarities between the probe media content and the reference media content are identified. A matching score given to the probe media content is weighted based on statistics regarding matches and false-positive rates for the content class of the reference media content. Further, classifiers can be trained on computed audio features and video features and/or video metadata and audio metadata of various media content.

Audio and video matching using a hybrid of fingerprinting and content based classification

Systems and methods are provided herein relating to real-time detection of inactive broadcasts during live stream ingestion. Both audio fingerprints and video fingerprints can be dynamically and continuously generated for a live stream ingestion. Sets of video fingerprints and sets of audio fingerprints can be continuously generated based on common successive overlapping time windows. A set of audio fingerprints and a set of video fingerprints can be associated with each time window. Video similarity scores and audio similarity scores can be generates for each time window to determine whether the stream is inactive or static during the time window. Only fingerprints relating to an active broadcast can be indexed in a fingerprint index.

Detection of inactive broadcasts during live stream ingestion

This disclosure relates to transformation invariant media matching. A fingerprinting component can generate a transformation invariant identifier for media content by adaptively encoding the relative ordering of interest points in media content. The interest points can be grouped into subsets, and stretch invariant descriptors can be generated for the subsets based on ratios of coordinates of interest points included in the subsets. The stretch invariant descriptors can be aggregated into a transformation invariant identifier. An identification component compares the identifier against a set of identifiers for known media content, and the media content can be matched or identified as a function of the comparison.

Transformation invariant media matching

A system and method for evaluating claims is disclosed. The system comprises a selection module, a query module, a communication module and a determination module. The selection module determines a review set including one or more claims based at least in part on claim data. The query module determines, based at least in part on the review set, an evaluation form including one or more queries associated with a first claim. The communication module receives answer data describing one or more answers responsive to the one or more queries included in the evaluation form. The determination module determines a validity decision associated with the first claim based at least in part on the answer data.

Claim evaluation system

An audio recognition service recognizes an audio sample across multiple content types. At least a partial set of results generated by the service are returned to a client while the audio sample is still being recorded and/or transmitted. The client additionally displays the results in real-time or near real-time to the user. The audio sample can be sent over a first HTTP connection and the results can be returned over a second HTTP connection. The audio recognition service further processes check-in selections received from the client for content items indicated by the results. Responsive to receiving the check-in selections, the service determines whether a user is eligible for a reward. If the user is eligible, the service provides the reward.

Real-time audio recognition using multiple recognizers

Systems and methods audio matching using interest point overlap are disclosed herein. The systems include determining at least one matching reference segment based on a probe segment. Interest points for both the at least one matching reference segment and the probe segment can be generated. Probe segment interest points and matching reference segment interest points can be time aligned and frequency aligned. A count can be generated based on a number of overlapping interest points between each set of reference interest points and the set of probe segment interest points. The disclosed systems and methods allow false positive reference to be identified and eliminated based on the count. The benefits in eliminating false positive matches improve the accuracy of an audio matching system.

Audio matching using time alignment, frequency alignment, and interest point overlap to filter false positives

Aspects relate to tagging portions of streaming media such that one or more actions can be taken on the tagged portions. An action can be to remove a section of the streaming media. Another action can be to retain a portion of the streaming media, regardless of whether or not other portions are retained. Another action can be to replace content with different content. The tagging can be facilitated by the user of a lightweight embedded watermark. In another example, the tagging can be facilitated through the use of watermark types.

Identifying portions of a media stream

Implementations are provided herein relating to audio matching. A variable length local sensitivity hash ('LSH') index can be created through a careful examination of existing LSH bands in the LSH index. LSH bands with offset lists that meet a band size threshold can be lengthened repeatedly until a maximum length threshold is reached or an offset list associated with a lengthened LSH band fails to meet the band size threshold. The LSH index can be further tuned by down-sampling or discarding LSH bands that reach a maximum length threshold and still lack discriminate properties.

Variable length local sensitivity hash index

Systems and methods are provided herein relating to audio matching. Descriptors can be generated based on anchor points and interest points that characterize the local neighborhood surrounding the anchor point. Characterizing the local spectrogram neighborhood surrounding anchor points can be more robust to pitch shift distortions and time stretch distortions. Those anchor points surrounded by a lack of spectral activity or even spectral activity can be filtered from further examination. Using these pitch shift and time stretch resistant audio features within descriptors can provide for more accurate and efficient audio matching.

Pitch shift and time stretch resistant audio matching

Down scoring overcrowded bands via IDF weighting scores provides a soft way to reduce the effect of common bands from Locality Sensitive Hashing (LSH) processes. An index component indexes live video references of a live streaming infrastructure pathway process in a reference index. A scoring component scores a set of bands with a set of inverse document frequency (IDF) weighting scores in the reference index. A high score is generated for bands that are featured in a small number of references and a low score is generated for bands featured in a high number of references.

IDF weighting of LSH bands for live reference ingestion

Identifying near identical versions of a probe sample from reference files comprises identifying discriminative regions of reference matches by generating a similarity matrix. The discriminative time frames are communicated to a client device and additional data associated with the probe sample can be retrieved having features of the discriminative regions. Based on the additional data, a single match can be generated to identify the probe sample.

Growing community of inventors

Zurich, Switzerland

Gheorghe Postelnicu