Microsoft Technology Licensing, LLC

Sheng Yi

Tianyi Chen

Xiao Tu

Yixin Shi

Biyi Fang

Reed L Townsend

Ivan Stojiljkovic

Kyle Thomas Beck

Nikola Nikolic

Yibo Sun

Jenna Hong

Jelena Mojasevic

Gilbert Antonius

Aman Orazaev

Megan L Fu

Gowtham Ganesan

Tra My Nguyen

Ryan Serrao

Apurva Sandeep Gandhi

Methods, systems, and computer program products are provided for signature verification. Signature verification may be provided for target signatures using genuine signatures. A signature verification model pipeline may extract features from a target signature and a genuine signature, encode and submit both to a neural network to generate a similarity score, which may be repeated for each genuine signature. A target signature may be classified as genuine, for example, when one or more similarity scores exceed a genuine threshold. A signature verification model may be updated or calibrated at any time with new genuine signatures. A signature verification model may be implemented with multiple trainable neural networks (e.g., for feature extraction, transformation, encoding, and/or classification).

Signature verification

The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. A sparsity-inducing regularization optimization process is performed on a machine learning model to generate a compressed machine learning model. A machine learning model is trained using a first set of training data. A sparsity-inducing regularization optimization process is executed on the machine learning model. Based on the sparsity-inducing regularization optimization process, a compressed machine learning model is received. The compressed machine learning model is executed to generate one or more outputs.

Model compression by sparsity-inducing regularization optimization

A computer system is provided that includes one or more processors configured to receive user input for inked content to a digital canvas, and process the inked content to determine one or more writing regions. Each writing region includes recognized text and one or more document layout features associated with that writing region. The one or more processors are further configured to tokenize a target writing region of the one or more writing regions into a sequence of tokens, process the sequence of tokens of the target writing region using a task extraction subsystem that operates on tokens representing both the recognized text and the one or more document layout features of the target writing region, segment the target writing region into one or more sentence segments, and classify each of the one or more sentence segments as a task sentence or a non-task sentence.

Sequence labeling task extraction from inked content

Model compression by sparsity&#x2014;inducing regularization optimization

Systems and methods for performing spatial line grouping on digital ink stokes. The system includes an electronic processor configured to access a set of hypothetical lines in an electronic document and determine a set of hypothetical line pairings. The electronic processor is also configured to determine, via a gradient boosting tree model, a merge confidence score for each hypothetical line pairing and compare a first merge confidence score with a merge threshold. The first merge confidence score is associated with a first hypothetical line and a first neighboring hypothetical line. The electronic processor is also configured to, in response to the first merge confidence score satisfying the merge threshold, merge the first hypothetical line and the first neighboring hypothetical line to form a first line grouping. The electronic processor is also configured to perform a digital ink stroke analysis on the electronic document based on the first line grouping.

Gradient boosting tree-based spatial line grouping on digital ink strokes

A data processing system for performing a semantic analysis of digital ink stroke data implements obtaining the digital ink stroke data representing handwritten text, drawings, or both; analyzing the digital ink stroke data to extract path signature feature information from the digital ink stroke data; analyzing the path signature feature information using a convolutional neural network (CNN) trained to perform a pixel-level sematic analysis of the digital ink stroke data and to output a pixel segmentation map with semantic prediction information for each pixel of digital ink stroke data; analyzing the pixel segmentation map to generate stroke-level semantic information using a pixel-to-stroke conversion model; and processing the digital ink stroke data based on the stroke-level semantic information.

Semantic segmentation for stroke classification in inking application

A spatially sparse convolutional neural network (CNN) framework is introduced to that leverages high sparsity of input data to significantly reduce the computational cost of applications that employ CNNs (e.g., inking applications and others) by avoiding unnecessary floating point mathematical operations. The framework, which is compatible with parallelized operations, includes (1) a data structure for sparse tensors that both (a) reduces storage burden and (b) speeds computations; (2) a set of sparse tensor operations that accelerate convolution computations; and (3) the merging of pooling and convolutional layers. Practical applications involving handwriting recognition and/or stroke analysis demonstrate a notable reduction in storage and computational burdens.

Spatially sparse convolutional neural networks for inking applications

An ink parsing system receives ink strokes at an inking device input and render the received ink strokes into an image in a pixel space. Writing strokes are detected in the image and labeled. Pixels corresponding to the labeled writing strokes are removed from the image. Drawing strokes in the image having the removed pixels are detected using and labeled. Writing objects and drawing objects corresponding, respectively, to the labeled writing strokes and the labeled drawing strokes are output. A digital ink parsing pipeline is thereby provided having accurate ink stroke detection and segmentation.

Object detection and segmentation for inking applications

Described herein is a system and method for visually enhancing digital ink of an electronic document. A trigger to visually enhance digital ink of portion(s) of the electronic document is received. In response to the received trigger, the digital ink of portion(s) of the electronic document to determine a semantic structure of the digital ink in response to the received trigger. The digital ink of the portion(s) of the electronic document are visually enhanced in accordance with the determined semantic structure. Visual enhancement can include horizontal line adjustment, aligning line(s), aligning word in a particular line using a baseline, adjusting vertical spacing of lines, paragraphs, and/or lists, adjusting spacing between words and/or list items in a particular line, modifying ink styling (e.g., ink size, ink thickness, ink color), adjusting sizing of characters in a same group, unifying ink color, and/or unifying ink thickness.

Visually enhanced digital ink

Methods, systems, and apparatuses for beautifying raw drawing data include a beautification engine having a raw data processor that receives the drawing data and identifies a set of segments in the data. A beautification director constructs a structural representation of the set of segments data that specifies a topological relationship between the segments. A single primitive beautifier identifies a segment in the structural representation to be beautified and a single primitive beautification action is applied to the segment to generate beautified raw drawing data. A multiple primitive beautifier identifies adjacent segments in the structural representation to be beautified and a multiple beautification action is applied to the adjacent segments to generate further beautified drawing data.

System and method for drawing beautification

Symbol recognition techniques may be applied to documents comprising various forms of content. Documents including both text and mathematical expressions may be problematic, as applying a recognizer that does not match the content may produce anomalous results. Instead, a parser may evaluate the document to classify respective regions as one of a text region or a mathematics region, based on the characteristics of each type of content. The recognizer corresponding to the content of each region may be applied to produce a composite document comprising both recognized text expressions and recognized mathematical expressions. Additional functionality may be presented based on the recognized content; e.g., text-based tools such as spell-check and natural-language translation may be applied to the text regions that comprise text expressions, and mathematics-based tools such as mathematical equation editors, mathematical expression solvers, and automated assignment grading may be applied to the mathematics regions that comprise mathematical expressions.

Growing community of inventors

Bellevue, WA, United States of America

Sheng Yi