Veritone, Inc.

Other

David Kettler

Peter Quoc Nguyen

Chad Steelberg

Karl Schwamb

Yu Zhao

Rather than randomly selecting neural networks to classify a media file, the conductor can determine which neural network engines (from the conductor ecosystem of neural networks) are the best candidates to classify a particular portion/segment of the media file (e.g., audio file, image file, video files). The best candidate neural network engine(s) can depend on the nature of the input media and the characteristics of the neural network engines. In object recognition and identification, certain neural networks can classify vehicles better than others, while another group of neural networks can classify structures better. The conductor can take out the guess work and construct in real-time an inter-classifier neural network using one or more layers selected from one or more pre-trained neural network, based on attribute(s) of the media file, to classify the media file.

Neural network orchestration

Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods includes receiving, from a first transcription engine, one or more transcription results of one or more audio segments of the multimedia file; identifying a first transcription result for a first audio segment having a low confidence of accuracy; identifying a first image data of the multimedia file corresponding to the first segment; receiving, from an image classification engine trained to classify image data, an image classification result of one or more portions of the first image data in response to requesting the image classification engine to classify the first image data; and selecting, based at least on the image classification result of the one or more portions of the first image data, a second transcription engine to re-classify the first audio segment.

System and method for neural network orchestration

Provided herein are embodiments of systems and methods for classifying one or more objects in an image. One of the methods includes: receiving object classification results of the image from one or more classification engines, the object classification results comprise classification of one or more objects and confidence scores associated with the one or more objects; aggregating the object classification results from the one or more classification engines to generate a list of confidence scores associated with each of the one or more objects, the list of confidence scores comprises one or more confidence scores from one or more classification engines; calculating an overall certainty score for each of the one or more objects based at least on the list of confidence scores; and generating a first orchestrated classification result based the overall certainty score for each of the one or more object.

Object detection machine learning

Methods and systems for training one or more neural networks for transcription and for transcribing a media file using the trained one or more neural networks are provided. One of the methods includes: segmenting the media file into a plurality of segments; extracting, using a first neural network, audio features of a first and second segment of the plurality of segments; and identifying, using a second neural network, a best-candidate engine for each of the first and second segments based at least on audio features of the first and second segments. A best-candidate engine is a neural network having a highest predicted transcription accuracy among a collection of neural networks.

Methods and systems for classifying a multimedia file using interclass data is disclosed. One of the methods can use classification results from one or more engines of different classes to select a different engine for the original classification task. For example, given an audio segment with associated metadata and image data, the disclosed interclass method can use the classification results from a topic classification of metadata and/or an image classification result of the image data as inputs for selecting a new transcription engine to transcribe the audio segment.

Growing community of inventors

Bellevue, WA, United States of America

David Kettler