Dropbox, Inc.

Columbia University

Other

University of California

Yale University

Kriegman-belhumeur Vision Technologies, LLC

Peter Neil Belhumeur

David J Kriegman

Thomas R Berg

Nils Peter Welinder

Jongmin Baek

Ying Xiong

Simon Kozlov

Neeraj Kumar

Shree Kumar Nayar

Bradley Neuberg

Leonard Fink

David W Jacobs

Todd Zickler

Dimitri Bitouk

Satya P Mallick

Michael Grossberg

Harish Peri

Athinodoros Georghiades

Alexander C Berg

The present disclosure is directed toward systems and methods to quickly and accurately identify boundaries of a displayed document in a live camera image feed, and provide a document boundary indicator within the live camera image feed. For example, systems and methods described herein utilize different display document detection processes in parallel to generate and provide a document boundary indicator that accurately corresponds with a displayed document within a live camera image feed. Thus, a user of the mobile computing device can easily see whether the document identification system has correctly identified the displayed document within the camera viewfinder feed.

Live document detection in a captured video stream

The present disclosure is directed toward systems and methods that efficiently and effectively generate an enhanced document image of a displayed document in an image frame captured from a live image feed. For example, systems and methods described herein apply a document enhancement process to a displayed document in an image frame that result in an enhanced document image that is cropped, rectified, un-shadowed, and with dark text against a mostly white background. Additionally, systems and method described herein determine whether a stored digital content item includes a displayed document. In response to determining that a stored digital content item does include a displayed document, systems and methods described herein generate an enhanced document image of a displayed document included in the stored digital content item.

Enhancing documents portrayed in digital images

The present disclosure relates to generating computer searchable text from digital images that depict documents utilizing an orientation neural network and/or text prediction neural network. For example, one or more embodiments detect digital images that depict documents, identify the orientation of the depicted documents, and generate computer searchable text from the depicted documents in the detected digital images. In particular, one or more embodiments train an orientation neural network to identify the orientation of a depicted document in a digital image. Additionally, one or more embodiments train a text prediction neural network to analyze a depicted document in a digital image to generate computer searchable text from the depicted document. By utilizing the identified orientation of the depicted document before analyzing the depicted document with a text prediction neural network, the disclosed systems can efficiently and accurately generate computer searchable text for a digital image that depicts a document.

Generating searchable text for documents portrayed in a repository of digital images utilizing orientation and text prediction neural networks

One or more embodiments of a digital content system allow a user to conveniently search and/or navigate through a collection of digital content items. In particular, a user can interact with a client device to search for and identify one or more digital content items within a collection of digital content items. For example, the digital content system may provide a photo from a collection of photos via a graphical user interface. The digital content system can receive a user input identifying a selection of one or more visual features within the photo. Based on the selected visual feature(s), the digital content system may identify photos from the collection of photos that include the identified visual feature(s) and provide access to the identified photos via the graphical user interface.

Navigating digital content using visual characteristics of the digital content

Computer-implemented techniques for sematic image retrieval. According to one technique, digital images are classified into N number of categories based on their visual content. The classification provides a set of N-dimensional image vectors for the digital images. Each image vector contains up to N number of probability values for up to N number of corresponding categories. An N-dimensional image match vector is generated that projects an input keyword query into the vector space of the set of image vectors by computing the vector similarities between a word vector for the input query and a word vector for each of the N number of categories. Vector similarities between the image match vectors and the set of image vectors can be computed to determine images semantically relevant to the input query.

Semantic image retrieval

Computer-implemented techniques for sematic image retrieval are disclosed. Digital images are classified into N number of categories based on their visual content. The classification provides a set of N-dimensional image vectors for the digital images. Each image vector contains up to N number of probability values for up to N number of corresponding categories. An N-dimensional image match vector is generated that projects an input keyword query into the vector space of the set of image vectors by computing the vector similarities between a word vector for the input query and a word vector for each of the N number of categories. Vector similarities between the image match vectors and the set of image vectors can be computed to determine images semantically relevant to the input query.

Systems and methods that efficiently and effectively generate an enhanced document image of a displayed document in an image frame captured from a live image feed are disclosed. For example, systems and methods described herein apply a document enhancement process to a displayed document in an image frame that result in an enhanced document image that is cropped, rectified, un-shadowed, and with dark text against a mostly white background. Additionally, systems and method described herein determine whether a stored digital content item includes a displayed document. In response to determining that a stored digital content item does include a displayed document, systems and methods described herein generate an enhanced document image of a displayed document included in the stored digital content item.

Growing community of inventors

New York, NY, United States of America

Peter Neil Belhumeur