A9.com, Inc.

Other

Amazon Technologies, Inc.

Xiaofan Lin

Arnab Sanat Kumar Dhua

Douglas Ryan Gray

Yu Lou

Adam Wiggen Kraft

Colin Jon Taylor

Sunil Ramesh

Atul Kumar

Kathy Wing Lam Ma

Simant Dube

Zhijiang Mark Lu

Jaishanker K Pillai

David Creighton Mott

Shashank Shekhar

Edward Hsiao

Mark Jay Nitzberg

Nityananda Jayadevaprakash

Son Dinh Tran

Shruti Sheorey

Arnab Sunat Kumar Dhua

Colin Jay Taylor

Aditya Deorha

A computing device can obtain data describing at least one document, the at least one document referencing at least one media object, wherein a portion of the at least one media object includes one or more characters. The computing device can obtain data describing the one or more characters in the at least one media object in the at least one document. The computing device can generate an updated copy of the at least one document that includes the data describing the one or more characters in the at least one media object. The computing device can present, on a display screen of the computing device and through an interface, the updated copy of the at least one document, wherein the one or more characters in the at least one media object are able to be selected or searched.

Media processing techniques for enhancing content

When a first search query including an image of an item is received to search for items associated with similar images, a second search query that includes text based on the image is generated. The text may be based on previous queries associated with the depicted item, visual features of the image, or text that is present in the image. The results from the first search query are scored based on their correspondence with the image of the item. Results having a score greater than a threshold are presented first in the output, followed by a selected number of results from the second search query. Results from the first search query that are associated with a score less than the threshold may be presented after the results from the second search query. This presentation increases the likelihood that items presented earlier in the output are relevant to the initial query.

Systems for determining image-based search results

Various embodiments enable a computing device to perform tasks such as processing an image to recognize text or an object in an image to identify a particular product or related products associated with the text or object. In response to recognizing the text or the object as being associated with a product available for purchase from an electronic marketplace, one or more advertisements or product listings associated with the product can be displayed to the user. Accordingly, additional information for the associated product can be displayed, enabling the user to learn more about and purchase the product from the electronic marketplace through the portable computing device.

Augmented reality recommendations

A user can capture an image of a text object of interest and have that image submitted for processing. The image can be pre-processed to improve quality and then submitted to an optical character recognition process to identify the words, characters, or strings in the image. At least some of these results can be submitted as a query to a search engine to obtain potential matches. In order to improve the accuracy of the results, information such as the titles for the results can be compared against each recognized word, character, or string from the image, including the ordering of those elements. An updated relevancy score can then be generated based on the full, ordered set. The recognized text is also analyzed to attempt to recognize model numbers or other identifiers that can be weighted more heavily as being indicative of accurate matches. Matches are selected from the re-ranked results.

Result re-ranking for object recognition

Approaches provide for analyzing image data to determine and/or recognize text in the image data. The recognized text can be used to generate a search query that can be automatically submitted to a search engine without having to type the search query to identify a product (or related products) associated with the image. For example, a camera of a computing device can be used to capture a live camera view (or single images) an item. An application executing on the computing device (or at least in communication with the computing device) can analyze the image data of the live camera view to determine a set of keywords (e.g., identified text) based on visual features extracted from the image data. The keywords can be used to query an index of product titles, common search queries, among other indexed data to return a ranked list of search suggestions based on a relevance function. The relevance function can consider the ordering of the keywords to rank search suggestions more highly that contain the keywords having the same word order. Further, the relevance function can consider the confidence of the visual recognition of each keyword, the confidence of each search suggestion, customer impact, as well as other factors to determine the ranking of the search suggestions. The search suggestions can be further refined to ensure search results that the user will be more likely to view and/or purchase.

Visual search suggestions

Approaches provide for recognizing and locating text represented in image data. For example, image data that includes representations of text can be obtained. A width-focused recognition engine can be configured to analyze the image data to determine a base-set of words. The base-set of words can be associated with logical structure information that describes a geometric relationship between words in the base-set of words. A set of bounding boxes that includes one or more base words can be determined, as well as a confidence value for each base word. A depth-focused recognition engine can be configured to analyze the image data to determine a focused-set of words, the focused-set of words associated with a set of bounding boxes and confidence values for respective words. A set of merged words can be determined from a set of overlapping bounding boxes that overlap a threshold amount. The set of merged words can include at least a portion of the base-set of words and/or the focused-set of words and are selected based at least in part on respective confidence values of words in the set of overlapping bounding boxes. Thereafter, a final set of words that includes the merged set of words and appended words can be determined.

Combination of heterogeneous recognizer for image-based character recognition

Various embodiments may increase scalability of image representations stored in a database for use in image matching and retrieval. For example, a system providing image matching can obtain images of a number of inventory items, extract features from each image using a feature extraction algorithm, and transform the same into their feature descriptor representations. These feature descriptor representations can be subsequently stored and used to compare against query images submitted by users. Though the size of each feature descriptor representation isn't particularly large, the total number of these descriptors requires a substantial amount of storage space. Accordingly, feature descriptor representations are compressed to minimize storage and, in one example, machine learning can be used to compensate for information lost as a result of the compression.

Scalable image matching

Systems and approaches are provided for optimizing self-portraiture. The background of the self-portrait can be enhanced by image registration or stitching techniques of images captured using one or more conventional cameras. Multiple standard resolution images can be stitched together to generate a panoramic or a composite image of a higher resolution. Foreground elements, such as one or more representations of users, can also be enhanced in various ways. The representations of the users can be composited to exclude undesirable elements, such as image data of one of the users extending her arm to capture the self-portrait. An ideal pose of the users can automatically be selected and other image enhancements, such as histogram optimization, brightness and contrast optimization, color-cast correction, or reduction or removal of noise, can automatically be performed to minimize user effort in capturing self-portraits.

Self-portrait enhancement techniques

Various approaches provide for detecting and recognizing text to enable a user to perform various functions or tasks. For example, a user could point a camera at an object with text, in order to capture an image of that object. The camera can be integrated with a portable computing device that is capable of taking the image and processing the image (or providing the image for processing) to recognize, identify, and/or isolate the text in order to send the image of the object as well as recognized text to an application, function, or system, such as an electronic marketplace.

Assisted text input for computing devices

Various embodiments enable the identification of semi-structured text entities in an imager. The identification of the text entities is a relatively simple problem when the text is stored in a computer and free of errors, but much more challenging if the source is the output of an optical character recognition (OCR) engine from a natural scene image. Accordingly, output from an OCR engine is analyzed to isolate a character string indicative of a text entity. Each character of the string is then assigned to a character class to produce a character class string and the text entity of the string is identified based in part on a pattern of the character class string.

Text entity recognition

Various embodiments enable a process to automatically attempt to select the most relevant words associated with products available for purchase from an electronic marketplace from an image frame. For example, an image frame containing text can be obtained and analyzed with an optical character recognition. The recognized words can then be preprocessed using various filtering and scoring techniques to narrow down a volume of text to a few relevant query terms. These query terms can then be sent to a search engine associated with the electronic marketplace to return relevant products to a user.

Text recognition for search results

Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

Growing community of inventors

Palo Alto, CA, United States of America

Xiaofan Lin