Microsoft Technology Licensing, LLC

Other

Ohad Jassin

Oron Nir

Royi Ronen

Avner Levi

Mor Geva Pipek

Ori Ziv

Olivier Nano

Ika Bar-Menachem

Milan Gada

Chin-Yew Lin

Eylon Ami

Abedelkader Asi

Ran Mizrachi

Daniel Nurieli

Liron Izhaki-Allerhand

Aspects of the technology described herein improve an object recognition system by specifying a type of picture that would improve the accuracy of the object recognition system if used to retrain the object recognition system. The technology described herein can take the form of an improvement model that improves an object recognition model by suggesting the types of training images that would improve the object recognition model's performance. For example, the improvement model could suggest that a picture of a person smiling be used to retrain the object recognition system. Once trained, the improvement model can be used to estimate a performance score for an image recognition model given the set characteristics of a set of training of images. The improvement model can then select a feature of an image, which if added to the training set, would cause a meaningful increase in the recognition system's performance.

Training set sufficiency for image analysis

In various embodiments, methods and systems for implementing a media management system, for video data processing and adaptation data generation, are provided. At a high level, a video data processing engine relies on different types of video data properties and additional auxiliary data resources to perform video optical character recognition operations for recognizing characters in video data. In operation, video data is accessed to identify recognized characters. A video OCR operation to perform on the video data for character recognition is determined from video character processing and video auxiliary data processing. Video auxiliary data processing includes processing an auxiliary reference object; the auxiliary reference object is an indirect reference object that is a derived input element used as a factor in determining the recognized characters. The video data is processed based on the video OCR operation and based on processing the video data, at least one recognized character is communicated.

Media management system for video data processing and adaptation data generation

Technology is disclosed for providing dynamic identification and extraction or tagging of contextually-coherent text blocks from an electronic document. In an embodiment, an electronic document may be parsed into a plurality of content tokens that each corresponds to a portion of the electronic document, such as a sentence or a paragraph. Employing a sliding window approach, a number of token groups are independently analyzed, where each group of tokens has a different number of tokens included therein. Each token group is analyzed to determine confidence scores for various determinable contexts based on content included in the token set. The confidence scores can then be processed for each token group to determine an entropy score for the token group. In this way, one of the analyzed token groups can be selected as a representative text block that corresponds to one of the plurality of determinable contexts. A corresponding portion of the electronic document can be tagged with a corresponding context determined based on the analyzed content included therein, and provided for output.

Dynamic extraction of contextually-coherent text blocks

Systems and methods are disclosed for inferring topics from a file containing both audio and video, for example a multimodal or multimedia file, in order to facilitate video indexing. A set of entities is extracted from the file and linked to produce a graph, and reference information is also obtained for the set of entities. Entities may be drawn, for example, from Wikipedia categories, or other large ontological data sources. Analysis of the graph, using unsupervised learning, permits determining clusters in the graph. Extracting features from the clusters, possibly using supervised learning, provides for selection of topic identifiers. The topic identifiers are then used for indexing the file.

Inferring topics with entity linking and ontological data

In various embodiments, methods and systems for implementing video segmentation are provided. A video management system implements a video segment manager that supports generating enhanced segmented video. Enhanced segmented video is a time-based segment of video content. Enhanced segmented video is generated based on a video content cognitive index, segmentation dimensions, segmentation rules and segment reconstruction rules. The video content cognitive index is built for indexing video content. Segmentation rules are applied to the video content to break the video content into time-based segments, the time-based segments are associated with corresponding segmentation dimensions for the video content. Segment reconstruction rules are then applied to the time-based segments to reconstruct the time-based segments into enhanced segmented video. The enhanced segmented video and corresponding values of the segmentation dimensions can be leveraged as distinct portions of the video content for different types of functionality in the video management system.

Video segmentation and searching by segmentation dimensions

In various embodiments, methods and systems for implementing video segment playlist generated are provided. A video management system implements a video segment manager that supports generating video segment playlists for video content. Generating a video segment playlist is based on enhanced segmented video associated with segmentation dimensions. An enhanced segmented video is a time-based segment of video content generated based on segmentation dimensions, segmentation rules and segment reconstruction rules. The video segment manager receives a search query. The video segment manager identifies a plurality of relevant enhanced segmented videos that satisfy the search query. A selection of at least a subset of the plurality of relevant enhanced segmented videos is received to generate a video segment playlist. The video segment manager generates the video segment playlist comprising references to the subset of the plurality of relevant enhanced segmented videos. The video segment playlist comprising the references is communicated and can be used for video playback.

Growing community of inventors

Tel Mond, Israel

Ohad Jassin