Ricoh Company, Ltd.

Ricoh Corporation

Pathwork Diagnostics, Inc.

Remington Partners

Jorge Moraleda

Jonathan J Hull

Berna Erol

Hidenobu Kishi

Kathrin Berkner

Jamey Graham

Qifa Ke

Michael J Gormish

Daniel G Van Olst

Anshul Garg

Sri Kaushik Pavani

Silvio Savarese

Ichiro Sakikawa

Ekta Prashnani

Kurt W Piersol

David Geoffrey Stork

Sadao Takahashi

Hector H Gonzalez-Banos

Ramya Narasimha

Sri Rama Prasanna Pavani

Takahiro Furukawa

Glenda G Anderson

Ljubomir J Buturovic

Emilio R Antunez

Srikrishna Bhat

Marco Antonio Covarrubias

Qifa Le

Landry Huet

MacGregor Belniak

The disclosure includes a system and method for creating, storing, and retrieving a focalized visualization related to a location, an event or a subject of interest. A visualization server receives a query for creating a visualization from a client device, identifies and retrieves one or more segments of a video stream satisfying the query, and generates the visualization based on the one or more segments of the video stream.

Focalized summarizations of a video stream

The disclosure includes a system and method for performing image rectification using a single image and information identified from the single image. An image recognition application receives an input image, identifies a plurality of objects in the input image, estimates rectification parameters for the plurality of objects, identifies a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects, estimates final rectification parameters based on the plurality of candidate rectification parameters, computes a global transformation matrix using the final rectification parameters, and performs image rectification on the input image using the global transformation matrix.

Single image rectification

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides systems, methods, and computer program products for modifying documents for shared use, and for collaborative discussion of shared documents. Captured digital images of documents or portions associated with a user are received, along with modifications to the images. Documents are recognized from the captured digital images, and the modifications to the images are applied to the documents. Alternatively, captured digital images of documents are received, and the documents are recognized along with a hotspot in the document. The user is authenticated to a website associated with the hotspot, and is provided access to an associated multimedia repository.

Indexed document modification sharing with mixed media reality

An MMR system integrating image tracking and recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a recognition result. Image tracking information also is provided for determining relative locations of images to each other. The mobile device includes an image tracker for providing at least a portion of the image tracking information. The disclosure also includes methods for image tracking-assisted recognition, recognition of multiple images using a single image query, and improved image tracking using MMR recognition.

Mixed media reality recognition with image tracking

The disclosure includes a system and method for distinguishing objects of similar appearance that vary in color. An image recognition application receives a query image of a plurality of objects, determines a candidate list of indexed images that match an object in the query image based on luminance features, crops an image of the object from the query image to generate a cropped image, computes color features for the object, ranks the candidate list of indexed images based on the color features, and provides the candidate list of indexed images to a user.

Color-based post-processing of images

A method and apparatus for using inertial navigation for obtaining calibration data for an indoor positioning system are disclosed. In one embodiment, the method comprises tracking motion of a device between known locations within a structure using inertial navigation, collecting a plurality of Wi-Fi signatures with the device while tracking the motion of the device between the known locations, and matching each of the plurality of Wi-Fi signatures to a point between the known locations, where the matched Wi-Fi signatures are for use in comparing to Wi-Fi signatures being received by the indoor positioning system to identify an indoor location of a device within the structure.

Using inertial navigation for calibration of indoor positioning system

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides a system, method, and computer program product for annotating unindexed documents. Captured digital images of unknown paper documents or portions associated with a user are received with annotations. Features are extracted from the captured digital image and are stored, along with the captured digital image and the annotations.

Document annotation sharing

An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. The MMR matching unit includes a distributor for distributing the image to corresponding content type specific index tables and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The techniques described herein also include a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.

Mixed media reality recognition using multiple specialized indexes

The disclosure includes a system and method for identifying multiple items in an image. A image recognition application receives a query image of items, computes features of the query image, for each feature finds an indexed image with closest matched features in a database, determines that the shape of the matched features is geometrically consistent, determines whether the query image matches the indexed image, responsive to the query image matching the indexed image, returns inliers, determines a region of interest where the match was found in the query image, removes inliers from the set of features to reduce the set of features and returns all matches found when the query image fails to match the indexed image.

Recognition procedure for identifying multiple items in images

An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit, and an MMR publisher. A computer includes a document authoring application, plug-in, and printer driver. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a method for capturing symbolic information from documents and for capturing printer dll functions.

Generating and storing a printed representation of a document on a local computer upon printing

An MMR system for publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher integrated into a network with an advertiser, an ad broker, and an MMR service bureau. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR service bureau uses the result to retrieve advertising or other links associated with the location on the page. The list of results and links are sent back to the MMR gateway for presentation on the mobile device. The present invention also includes a number of novel methods including a differentially weighting links associated with an MMR document.

Mixed media reality retrieval of differentially-weighted links

The present invention relies on the two-dimensional information in documents and encodes two-dimensional structures into a one-dimensional synthetic language such that two-dimensional documents can be searched at text search speed. The system comprises: an indexing module, a retrieval module, an encoder, a quantization module, a retrieval engine and a control module coupled by a bus. Electronic documents are first indexed by the indexing module and stored as a synthetic text library. The retrieval module then converts an input image to synthetic text and searches for matches to the synthetic text in the synthetic text library. The matches can be in turn used to retrieve the corresponding electronic documents. In one or more embodiments, the present invention includes a method for comparing the synthetic text to documents that have been converted to synthetic text for a match.

Retrieving electronic documents by converting them to synthetic text

An MMR system integrating image tracking and recognition comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. Image tracking information also is provided for determining relative locations of images within a document page. The mobile device includes an image tracker for providing at least a portion of the image tracking information. The present invention also includes methods for image tracking-assisted recognition, recognition of multiple images using a single image query, and improved image tracking using MMR recognition.

The mobile device includes a client that has a number of modules, and the MMR Gateway and MMR matching unit are implemented as a server that has a number of modules. The implementation of the MMR system as a client and a server is advantageous because the modules may be distributed among the client and the server in a variety of configurations. The present invention includes a capture module, a preprocessing module, a feature extraction module, a retrieval module, a send message module, an action module, a prediction module, a feedback module, a sending module, an MMR database, a streaming module, an e-mail module, a voice recognition system and an audio database. These modules and systems are operational upon the client or the server.

Growing community of inventors

Menlo Park, CA, United States of America

Jorge Moraleda