Microsoft Technology Licensing, LLC

Djordje Nijemcevic

Milan Vugdelija

David Nister

Magdalena Vukosavljevic

Bodin Dresevic

Christopher Maurice Mei

Georg F Petschnigg

Vaibhav Thukral

Ruchi Bhargava

Matthew Sammis Ashman

Ivan Stojiljkovic

Ruchita Bhargava

Sasa Galic

Pavle Josipović

Ana Mitrovic

Chandra Sekhar Gatla

Derek L Knee

Gerhard A Schneider

John Eldridge

Robert W Havlik

Aleksandar Antonijevic

Ronald Boskovic

Ivan Mitic

Mircea Cimpoi

Embodiments that relate to determining an estimated pupil region of an eye are disclosed. In one embodiment a method includes receiving an image of an eye, with the image comprising a plurality of pixels. A rough pupil region is generated using at least a subset of the plurality of pixels. A plurality of pupil boundary point candidates are extracted from the rough pupil region, with each of the candidates weighted based on color values of at least two neighbor pixels. A parametric curve may be fitted to the weighted pupil boundary point candidates to determine the estimated pupil region of the eye of the user.

Pupil detection

The technology described herein employs techniques for overlaying (superimposing) translated text on top of (over) scanned text in realtime. The technology recognizes text in an image and identifies a first language in which the text is written. The technology translates the text into a second language and generates an overlay in the second language. The overlay is then inserted into the display over the text in the image.

Text overlay techniques in realtime translation

Various embodiments related to entering text into a computing device via eye-typing are disclosed. For example, one embodiment provides a method that includes receiving a data set including a plurality of gaze samples, each gaze sample including a gaze location and a corresponding point in time. The method further comprises processing the plurality of gaze samples to determine one or more likely terms represented by the data set.

Eye-typing term recognition

The technology disclosed herein provides various embodiments for mapping glints that reflect off from an object to light sources responsible for the glints. Embodiments disclosed herein are able to correctly map glints to light sources by capturing just a few images with a camera. Each image is captured while illuminating the object with a different pattern of light sources. A glint free image may also be determined. A glint free image is one in which the glints have been removed by image processing techniques.

Mapping glints to light sources

Architecture that employs techniques for overlaying (superimposing) translated text on top of (over) scanned text in realtime translation to provide clear visual correlation between original text and translated text. Algorithms are provided that overlay text in cases of translated scanned text of language written in first direction to a language written in same direction, translate scanned text from a first language written in a first direction to a second language written in the opposite direction, and translated scanned text from a language written in a first direction to language written in a different direction.

Architecture that automatically computes if a quadrangular object captured in a given image is or is not a markable board (e.g., a whiteboard, green board, chalkboard, etc.). The markable board has a surface on which marks can be applied using chalk, ink, dry ink, or any other suitable marking instrument or tool for the given surface. The imaged quadrangular object can be defined as having a background image and a foreground image. The background image is representative of a markable board with no applied surface marks and the foreground image comprises all discernible marks applied to the board surface, but does not include the background image. A set of performance-friendly features is received and processed by a machine-learning classifier to compute if the given quadrangular object is a markable board. Thereafter, if the given image is determined to be a markable board, image enhancement is performed.

Automatic classification and color enhancement of a markable surface

Embodiments of camera-based scanning are described. In various embodiments, one or more quadrangular objects are automatically selected from a captured image for scanning. The automatic selection is determined to be successful based on the selected quadrangular objects having an associated score that exceeds a predefined threshold. One or more scanned documents are created from portions of the captured image corresponding to the selected quadrangular objects, and the created scanned documents include corrections for perspective distortions of the selected quadrangular objects in the captured image.

Camera-based scanning

An image processing apparatus is provided that includes a character chopper component that segments words into individual characters in a bitmap of a textual image undergoing an OCR process. The Character chopper component is configured to produce a set of (possibly curved) chop-lines which divide a bitmap of any given word into its individual character or glyph candidates. Cases where an input bitmap contains two separate words are handled by marking a place where those words should be split. The character segmentation algorithm computes the set of vertically oriented, curved chop-lines by considering glyph and background colors in a given word bitmap. The set is filtered afterwards using various heuristics, in order to preserve those lines that indeed do separate a word's glyphs and minimize the number of those that do not.

Segmentation of a word bitmap into individual characters or glyphs during an OCR process

A method for enhancing a textual image for undergoing optical character recognition begins by receiving an image that includes native lines of text. A background line profile is determined which represents an average background intensity along the native lines in the image. Likewise, a foreground line profile is determined which represents an average foreground background intensity along the native lines in the image. The pixels in the image are assigned to either a background or foreground portion of the image based at least in part on the background line profile and the foreground line profile. The intensity of the pixels designated to the background portion of the image is adjusted to a maximum brightness so as to represent a portion of the image that does not include text.

Text enhancement of a textual image undergoing optical character recognition

A method for identifying words in a textual image undergoing optical character recognition includes receiving a bitmap of an input image which includes textual lines that have been segmented by a plurality of chop lines. The chop lines are each associated with a confidence level reflecting a degree to which the respective chop line properly segments the textual line into individual characters. One or more words are identified in one of the textual lines based at least in part on the textual lines and a first subset of the plurality of chop lines which have a chop line confidence level above a first threshold value. If the first word is not associated with a sufficiently high word confidence level, at least a second word in the textual line is identified based at least in part on a second subset of the plurality of chop lines which have a confidence level above a second threshold value lower than the first threshold value.

Word recognition of text undergoing an OCR process

Embodiments of camera-based scanning are described. In various embodiments, scanned documents can be created using images captured by a camera associated with a device. An image captured by the camera is processed to identify portions within the image that correspond to rectangular objects such as paper, business cards, whiteboards, screens, and so forth. One or more of these portions can be selected for scanning automatically based on a scoring scheme and/or semi-automatically with the aid of input from a user. One or more scanned documents are created from the selected portions by un-warping the selected portions to remove effects of perspective (e.g., rectify the portions to rectangles) and applying various image enhancements to improve appearance.

An optical character recognition process characterizes text lines in a textual image by their base-line, mean-line and x-height. The base-line for at least one text line in the image is determined by finding a parametric curve that maximizes a first fitness function that depends on the values of pixels through which the parametric curve passes and pixels below the parametric curve. The base-line corresponds to the parametric curve for which the first fitness function is maximized. The first fitness function is designed so that it increases with increasing lightless or brightness of pixels immediately below the parametric curve while also increasing with decreasing lightness of pixels through which the parametric curve passes. The mean-line is determined by incrementally shifting the base-line upward by predetermined amounts (e.g., a single pixel) until a second fitness function for the shifted base-line is maximized. The second fitness function is essentially the inverse of the first fitness function. Specifically, the second fitness function increases with increasing lightless of pixels immediately above the shifted base-line while also increasing with decreasing lightness of pixels through which the shifted base-line passes. The x-height is equal to the sum of the predetermined amounts by which the base-line is shifted upward in order to maximize the second fitness function. In some cases different groups of text-lines in the textual image may be characterized differently from one another. For example, each group may be characterized by a most probable x-height for that group.

Resolution adjustment of an image that includes text undergoing an OCR process

An image deskew system and techniques are used in the context of optical character recognition. An image is obtained of an original set of characters in an original linear (horizontal) orientation. An acquired set of characters, which is skewed relative to the original linear orientation by a rotation angle, is represented by pixels of the image. The rotation angle is estimated, and a confidence value may be associated with the estimation, to determine whether to deskew the image. In connection with rotation angle estimation, an edge detection filter is applied to the acquired set of characters to produce an edge map, which is input to a linear hough transform filter to produce a set of output lines in parametric form. The output lines are assigned scores, and based on the scores, at least one output line is determined to be a dominant line with a slope approximating the rotation angle.

Growing community of inventors

Belgrade, Republic of Serbia

Djordje Nijemcevic