Columbia University

AT&T Intellectual Property I, LP

Other

Mitsubishi Electric Research Laboratories, Inc.

Eastman-kodak Company

Amazon Technologies, Inc.

University of California

At+t Corp.

Ibm Thomas J. Watson Search Center

International Business Machines Corporation

Shih-Fu Chang

Ana Belen Benitez

Atul Puri

Qian Huang

Seungyup Paek

Chung-Sheng Li

John Richard Smith

Ajay Divakaran

Horace J Meng

Lawrence David Bergman

Alexander C Loui

Huifang Sun

Paul Sajda

Wei Jiang

Jun Wang

David Jangraw

Hongzhi Li

Joseph G Ellis

Dongqing Zhang

Daniel Morozoff-Abezgauz

Svebor Karaman

Seungup Paek

Lexing Xie

Alejandro Jaimes

Ching-Yung Lin

Peng Xu

Eric Pohlmeyer

Barbara Hanna

Brendan Jou

Eric Martin Zavesky

Anthony Vetro

Amy Ruth Reibman

Kyeongok Kang

Gustavo De Los Reyes

Yong Wang

Jae-Gon Kim

Lyndon Kennedy

Henry H Wu

Pradeep Natarajan

Justin Che-I Chuang

Jinwoong Kim

Daniel Ellis

Yue Wu

Xu Zhang

Courtenay Cotton

Junfeng He

Jianhao Meng

Tai-Hsu Lin

Paek Seungyup

Di Zhong

Hari Sundaram

William Chen

Shahram Ebadollahi

Varsha Hedau

Wen-Lung Chen

David Messerschmitt

John R Smith

Charlie Judice

Alexandros Elefteriadis

An image processing system may receive image data from a camera of a user device and perform gaze prediction processing of the image data to predict one or more gaze patterns. The gaze prediction processing may include processing the image data using a neural network to detect faces and/or objects and generate an image feature map. The gaze prediction processing may include performing gaze direction prediction operations using the feature map and detected faces and/or objects to determine gaze direction probability data. The gaze prediction processing may include predicting a gaze pattern based on the gaze direction probability data and the image feature map. The gaze pattern may be short-term (e.g., atomic-level) or long-term (e.g., event-level).

Gaze prediction

A system for aggregation and organization of multimedia data acquired from a plurality of sources can include a data collection element, a multimedia segmentation element, a memory, an extraction element, and a user interface. The data collection element can be configured to receive multimedia data. The multimedia segmentation element can be coupled to the data collection element and can be configured to separate the multimedia data into a plurality of multimedia data segments. The memory can be configured to store hierarchical category information having a plurality of categories. The extraction element can be coupled to the multimedia segmentation element and the memory and can be configured to associate at least one of the plurality of categories with each of the plurality of multimedia data segments.

Methods and systems for aggregation and organization of multimedia data acquired from a plurality of sources

Human visual perception is able to recognize a wide range of targets but has limited throughput. Machine vision can process images at a high speed but suffers from inadequate recognition accuracy of general target classes. Systems and methods are provided that combine the strengths of both systems and improve upon existing multimedia processing systems and methods to provide enhanced multimedia labeling, categorization, searching, and navigation.

Rapid image annotation via brain state decoding and visual pattern mining

The invention provides a system and method for integrating multimedia descriptions in a way that allows humans, software components or devices to easily identify, represent, manage, retrieve, and categorize the multimedia content. In this manner, a user who may be interested in locating a specific piece of multimedia content from a database, Internet, or broadcast media, for example, may search for and find the multimedia content. In this regard, the invention provides a system and method that receives multimedia content and separates the multimedia content into separate components which are assigned to multimedia categories, such as image, video, audio, synthetic and text. Within each of the multimedia categories, the multimedia content is classified and descriptions of the multimedia content are generated. The descriptions are then formatted, integrated, using a multimedia integration description scheme, and the multimedia integration description is generated for the multimedia content. The multimedia description is then stored into a database. As a result, a user may query a search engine which then retrieves the multimedia content from the database whose integration description matches the query criteria specified by the user. The search engine can then provide the user a useful search result based on the multimedia integration description.

Multimedia integration description scheme, method and system for MPEG-7

A system and method is provided for editing and parsing compressed digital information. The compressed digital information may include visual information which is edited and parsed in the compressed domain. In a preferred embodiment, the present invention provides a method for detecting moving objects in a compressed digital bitstream which represents a sequence of fields or frames of video information for one or more captured scenes of video.

Methods and architecture for indexing and editing compressed video over the world wide web

The present invention relates to systems and methods for motion estimation and mode decision for low-complexity H.264 standard decoders. The present invention includes a method for optimizing the selection of motion vectors and motion compensation block modules in a video encoder in order to decrease the complexity of the video upon decoding. The novel method of the present invention may include novel steps for selecting motion vectors, block modes, and for applying a complexity-control algorithm to encode the received input video data sequence in accordance with the identified target complexity level. The present invention may be implemented in accordance with current and future video decoding standards to optimize decoding by reducing decoding complexity and thereby reducing required resources and power consumption.

System and method for motion estimation and mode decision for low-complexity H.264 decoder

Determining ranked candidate media in response to media query data corresponding to a query media includes receiving the media query data including feature data of the query media, coordinate data, and boundary data, matching the features with corresponding features of an media database using the feature data to identify features in the media database within a predetermined hamming distance in a hash table from the corresponding features of the query media to obtain matched features in the media database, determining candidate media whose number of matched features exceeds a matched feature threshold, generating a geometry similarity score between the query media and each candidate media using the feature data and the coordinate data, generating a boundary similarity score between the query media and each candidate media using the boundary data, ranking the candidate media based on the numbers of matched features, the geometry similarity scores and the boundary similarity scores.

Systems and methods for mobile search using Bag of Hash Bits and boundary reranking

Systems and methods are described for determining manipulation history among a plurality of images. The described techniques include selecting a pair of images from the plurality of images, detecting one or more manipulations operable to transform one of the images to the other, and based on the manipulations detected, determining a parent-child relationship between the pair or pairs of images. The described techniques can further include repeating the selecting two images, detecting manipulations, and determining the parent-child relationship for each pairs of images in the plurality of images, constructing a visual migration map for the images, and presenting the visual migration map in a user readable format.

Systems and methods for image archaeology

Caption boxes which are embedded in video content can be located and the text within the caption boxes decoded. Real time processing is enhanced by locating caption box regions in the compressed video domain and performing pixel based processing operations within the region of the video frame in which a caption box is located. The captions boxes are further refined by identifying word regions within the caption boxes and then applying character and word recognition processing to the identified word regions. Domain based models are used to improve text recognition results. The extracted caption box text can be used to detect events of interest in the video content and a semantic model applied to extract a segment of video of the event of interest.

System and method for extracting text captions from video and generating video summaries

Systems and methods for describing video content establish video description records which include an object set (), an object hierarchy () and entity relation graphs (). Video objects can include global objects, segment objects and local objects. The video objects are further defined by a number of features organized in classes, which in turn are further defined by a number of feature descriptors (, and). The relationships () between and among the objects in the object set () are defined by the object hierarchy () and entity relation graphs (). The video description records provide a standard vehicle for describing the content and context of video information for subsequent access and processing by computer applications such as search engines, filters and archive systems.

Video description system and method

Systems and methods for searching a database of media content wherein the user can dynamically and interactively perform searches and navigate search results. One or more search anchors are received, and at least one of the search anchors is associated with an anchor cell on a navigation map. One or more documents assigned to at least one cell on the navigation map can be determined, and the cells are populated with search results based at least in part on the search anchors. At least one of the documents is then displayed to a user.

System and method for dynamically and interactively searching media data

Techniques for generating utility-based descriptors from compressed multimedia information are disclosed. A preferred method includes the steps of receiving least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each the portions of utility based descriptor information to generate adapted multimedia segments, using a quality management method to generate measurement for each adapted multimedia segment, and generating a utility based descriptors based on the portions of utility based descriptor information and corresponding quality measurements.

Method and system for optimal video transcoding based on utility function descriptors

Growing community of inventors

New York, NY, United States of America

Shih-Fu Chang