Microsoft Technology Licensing, LLC

Microsoft Corporation

Cha Zhang

Zhengyou Zhang

Jin Li

Zicheng Liu

Philip Andrew Chou

Yong Rui

Paul A Viola

Dinei A Florencio

Dinei Afonso Ferreira Florencio

Ross Garrett Cutler

Rajesh Hegde

Yunnan Wu

Christian F Huitema

Sasa Junuzovic

Xinding Sun

Pei Yin

Yu-Yun Dai

Timothy Andrew Large

Li-wei He

Lin Liang

Stanley Thomas Birchfield

Avronil Bhattacharjee

Adam G Kirk

Ruigang Yang

Nikolay Smolyanskiy

Oliver Arthur Whyte

Sean Eron Anderson

Gil Moshe Nahmias

Adarsh Prakash Murthy Kowdle

Xuedong D Huang

Michael F Cohen

Alastair Wolman

Gang Hua

Jason Matthew Cahill

Wenwu Zhu

Eyal Krupka

Ying Shan

Kaushik Chakrabarti

Jonathan Alan Handler

Igor Kviatkovsky

Yinpeng Chen

Shujie Liu

Nirupama Chandrasekaran

Michael Todd Gillam

Igor Abramovski

Timothy R O'Connor

Xiaogang Wang

Myung-suk Song

Craig Frederick Feied

Jianwen Zhang

Renato Reder Cazangi

Bin Yu

Demba Ba

Linjie Luo

John Christopher Gillotte

John Richard Corring

Yuet Ching

Neta Haiby

Vinod Kurpad

Souvik Kundu

Mengya Hu

Alexander T Gorevski

Zheng Chen

Leon Romaniuk

Sean Louis Goldberg

Bernardino Romera Paredes

Anatoly Yevgenyevich Ponomarev

Ting Yu

Shih Chia Wang

Yijuan Lu

Lynsey Liu

Andrei A Gaivoronski

Yu Cheng

Dinei Florencio

The disclosure herein describes generating input key-standard key mappings for a form. A set of input key-value pairs are received, and a subset of candidate form types are determined from a set of form types using the input key-value pairs. A set of standard keys associated with the determined subset of candidate form types are obtained. A set of input key-standard key pairs are generated using the set of input key-value pairs and the obtained set of standard keys and the set of input key-standard key pairs are narrowed using a narrowing rule. Ranking scores for each input key-standard key pair of the narrowed set of input key-standard key pairs are generated. Each input key of the set of input key-vale pairs is mapped to a standard key of the set of standard keys using at least the generated ranking scores of the narrowed set of input key-standard key pairs.

Knowledge driven pre-trained form key mapping

The disclosure herein describes providing signature data of an input document. Text data of the input document is obtained (e.g., OCR data generated from image data) and a first set of signature fields are identified using signature key-value pairs of the text data. A first subset of signed signature fields and a first subset of unsigned signature fields are determined based on mapping to a set of predicted values. A second set of signature fields are determined using a region prediction model applied to image data of the input document. Region images associated with the first subset of unsigned signature fields and with second set of signature fields are obtained and a second set of signed signature fields and a second set of unsigned signature fields are determined using a signature recognition model. Signature output data is provided including signed signature fields and/or unsigned signature fields.

Entry detection and recognition for custom forms

Interfaces and systems are provided for harvesting ground truth from forms to be used in training models based on key-value pairings in the forms and to later use the trained models to identify related key-value pairings in new forms. Initially, forms are identified and clustered to identify a subset of forms to label with the key-value pairings. Users provide input to identify keys to use in labeling and then select/highlight text from forms that are presented concurrently with the keys in order to associate the highlighted text with the key(s) as the corresponding key-value pairing(s). After labeling the forms with the key-value pairings, the key-value pairing data is used as ground truth for training a model to independently identify the key-value pairing(s) in new forms. Once trained, the model is used to identify the key-value pairing(s) in new forms.

Enhanced supervised form understanding

The disclosed technology is generally directed to optical character recognition for forms. In one example of the technology, optical character recognition is performed on a plurality of forms. The forms of the plurality of forms include at least one type of form. Anchors are determined for the forms, including corresponding anchors for each type of form of the plurality of forms. Feature rules are determined, including corresponding feature rules for each type of form of the plurality of forms. Features and labels are determined for each form of the plurality of forms. A training model is generated based on a ground truth that includes a plurality of key-value pairs corresponding to the plurality of forms, and further based on the determined features and labels for the plurality of forms.

Supervised OCR training for custom forms

The disclosed technology is generally directed to optical text recognition for forms. In one example of the technology, line grouping rules are generated based on the generic forms and a ground truth for the generic forms. Line groupings are applied to the generic forms based on the line grouping rules. Feature extraction rules are generated. Features are extracted from the generic forms based on the feature extraction rules. A key-value classifier model is generated, such that the key-value classifier model is configured to determine, for each line of a form: a probability that the line is a value, and a probability that the line is a key. A key-value pairing model is generated, such that the key-value pairing model is configured to predict, for each key in a form, which value in the form corresponds to the key.

Unsupervised domain adaptation from generic forms for new OCR forms

Various examples related to determining a location of an active participant are provided. In one example, image data of a room from an image capture device is received. First audio data from a first microphone array at the image capture device is received. Second audio data from a second microphone array spaced from the image capture device is received. Using a three dimensional model, a location of the second microphone array is determined. Using the first audio data, second audio data, location of the second microphone array, and an angular orientation of the second microphone array, an estimated location of the active participant is determined.

Active speaker location detection

A three-dimensional shape parameter computation system and method for computing three-dimensional human head shape parameters from two-dimensional facial feature points. A series of images containing a user's face is captured. Embodiments of the system and method deduce the 3D parameters of the user's head by examining a series of captured images of the user over time and in a variety of head poses and facial expressions, and then computing an average. An energy function is constructed over a batch of frames containing 2D face feature points obtained from the captured images, and the energy function is minimized to solve for the head shape parameters valid for the batch of frames. Head pose parameters and facial expression and animation parameters can vary over each captured image in the batch of frames. In some embodiments this minimization is performed using a modified Gauss-Newton minimization technique using a single iteration.

Computing 3D shape parameters for face animation

The described implementations relate to enhancement images, such as in videoconferencing scenarios. One system includes a poriferous display screen having generally opposing front and back surfaces. This system also includes a camera positioned proximate to the back surface to capture an image through the poriferous display screen.

Imaging through a display screen

Techniques for automatically connecting to a service controller are described herein. In one example, a service controller device includes a processor and a computer-readable memory storage device storing executable instructions that cause the processor to broadcast at least one of an access credential, connection information or an access credential hash embedded in an audio signal. The processor can also authenticate a client device based on a transmission of at least one of the connection information, the access credential, or the access credential hash from the client device to the client connector and transmit data to the client device in response to authenticating the client device.

Audio based discovery and connection to a service controller

The description relates to facial tracking. One example can include an orientation structure configured to position the wearable device relative to a user's face. The example can also include a camera secured by the orientation structure parallel to or at a low angle to the user's face to capture images across the user's face. The example can further include a processor configured to receive the images and to map the images to parameters associated with an avatar model.

Facial expression tracking

Multi-view video that is being streamed to a remote device in real time may be encoded. Frames of a real-world scene captured by respective video cameras are received for compression. A virtual viewpoint, positioned relative to the video cameras, is used to determine expected contributions of individual portions of the frames to a synthesized image of the scene from the viewpoint position using the frames. For each frame, compression rates for individual blocks of a frame are computed based on the determined contributions of the individual portions of the frame. The frames are compressed by compressing the blocks of the frames according to their respective determined compression rates. The frames are transmitted in compressed form via a network to a remote device, which is configured to render the scene using the compressed frames.

Multi-view video compression and streaming based on viewpoints of remote viewer

Various examples related to determining a location of an active speaker are provided. In one example, image data of a room from an image capture device is received and a three dimensional model is generated. First audio data from a first microphone array at the image capture device is received. Second audio data from a second microphone array laterally spaced from the image capture device is received. Using the three dimensional model, a location of the second microphone array with respect to the image capture device is determined. Using the audio data and the location and angular orientation of the second microphone array, an estimated location of the active speaker is determined. Using the estimated location, a setting for the image capture device is determined and outputted to highlight the active speaker.

Described is a technology by which medical patient facial images are acquired and maintained for associating with a patient's records and/or other items. A video camera may provide video frames, such as captured when a patient is being admitted to a hospital. Face detection may be employed to clip the facial part from the frame. Multiple images of a patient's face may be displayed on a user interface to allow selection of a representative image. Also described is obtaining the patient images by processing electronic documents (e.g., patient records) to look for a face pictured therein.

Automated acquisition of facial images

A depth construction module is described that receives depth images provided by two or more depth capture units. Each depth capture unit generates its depth image using a structured light technique, that is, by projecting a pattern onto an object and receiving a captured image in response thereto. The depth construction module then identifies at least one deficient portion in at least one depth image that has been received, which may be attributed to overlapping projected patterns that impinge the object. The depth construction module then uses a multi-view reconstruction technique, such as a plane sweeping technique, to supply depth information for the deficient portion. In another mode, a multi-view reconstruction technique can be used to produce an entire depth scene based on captured images received from the depth capture units, that is, without first identifying deficient portions in the depth images.

Depth reconstruction using plural depth capture units

A system described herein includes a receiver component that receives a first digital image from a color camera, wherein the first digital image comprises a planar object, and a second digital image from a depth sensor, wherein the second digital image comprises the planar object. The system also includes a calibrator component that jointly calibrates the color camera and the depth sensor based at least in part upon the first digital image and the second digital image.

Calibration between depth and color sensors for depth cameras

Growing community of inventors

Sammamish, WA, United States of America

Cha Zhang