Intuit, Inc.

Apple Inc.

Sreeneel K Maddika

Vijay S Yellapragada

Eugene Krivopaltsev

Peijun Chiang

Chaitanya Mannemala

Vijay Yellapragada

Garrett L Weinberg

Sammy Lee

Srikanth Vishnubhotla

Grace Pariante

Andrew B Firstenberger

Ahmed Serag El Din Hussen Abdelaziz

Bobby G Bray, Jr

Stephen Olivier Lemay

Pedro Mari

Devang Kalidas Naik

Aswath Manoharan

Jay Moon

Keith P Avery

Lynn I Streja

Sarmad Aziz Malik

Myra C Lukens

Karan M Daryanani

Abhishek Rawat

Bobby G Bray

Bobby G Bray Jr

An example process includes: while displaying a portion of an extended reality (XR) environment representing a current field of view of a user: detecting a user gaze at a first object displayed in the XR environment, where the first object is persistent in the current field of view of the XR environment; in response to detecting the user gaze at the first object, expanding the first object into a list of objects including a second object representing a digital assistant; detecting a user gaze at the second object; in accordance with detecting the user gaze at the second object, displaying a first animation of the second object indicating that a digital assistant session is initiated; receiving a first audio input from the user; and displaying a second animation of the second object indicating that the digital assistant is actively listening to the user.

Extended reality based digital assistant interactions

Systems and processes for operating an intelligent automated assistant are provided. For example, a first speech input is received from a user. In response to receiving the first speech input, a response is provided. A first output is provided corresponding to a digital assistant in a first state, and a second speech input is received from the user. A first plurality of values is obtained. Based on the first plurality of values, a first confidence level corresponding to the second speech input is obtained. In accordance with a determination that the first confidence level exceeds a first threshold confidence level, a second output is provided corresponding to the digital assistant in a second state. The second speech input continues to be received.

Multiple state digital assistant for continuous dialog

Systems and processes for operating an intelligent automated assistant are provided. An examples process of operating an intelligent automated assistant includes, at an electronic device with one or more processors and memory, receiving audio input, determining a direct-to-reverberant energy ratio based on the audio input, and determining a head pose of a user based on the direct-to-reverberant energy ratio.

Determining head pose based on room reverberation

The present disclosure relates to optical character recognition using captured video. According to one embodiment, using a first image in stream of images depicting a document, the device extracts text data in a portion of the document depicted in the first image and determines a first confidence level regarding an accuracy of the extracted text data. If the first confidence level satisfies a threshold value, the device saves the extracted text data as recognized content of the source document. Otherwise, the device extracts the text data from the portion of the document as depicted in one or more second images in the stream and determines a second confidence level for the text data extracted from each second image until identifying one of the second images where the second confidence level associated with the text data extracted from the identified second image satisfies the threshold value.

Optical character recognition (OCR) accuracy by combining results across video frames

Techniques are disclosed for performing optical character recognition (OCR) by identifying a template based on a hash of a document. One embodiment includes a method for identifying a template associated with an image. The method includes receiving a digital image, a portion of the image depicting a first document, and extracting the portion of the image. The method further includes scaling the portion of the image and generating a first hash from the scaled image. The method further includes comparing the first hash to a set of hashes, each corresponding to a template. The method further includes selecting a first template as corresponding to the first document based on comparing the first hash to the set of hashes and extracting one or more sections of the portion of the image based on the selected first template. The method further includes performing OCR on the extracted one or more sections.

Optical character recognition utilizing hashed templates

Systems of the present disclosure generate accurate training data for optical character recognition (OCR). Systems disclosed herein generates images of a text passage as displayed piecemeal in a user interface (UI) element rendered in a selected font type and size, determine accurate dimensions and locations of bounding boxes for each character pictured in the images, stitch together a training image by concatenating the images, and associate the training image, the bounding box dimensions and locations, and the text passage together in a collection of training data. The collection of training data also includes a computer-readable master copy of the text passage with newline characters inserted therein.

Automating creation of accurate OCR training data using specialized UI application

Aspects of the present disclosure provide methods and apparatuses for detecting duplicate copies of a form in an image of a document. An exemplary method generally includes obtaining a first digital image of a document, performing one or more transformations on the first digital image, determining one or more rectangles in the transformed first digital image, identifying at least a first duplicate copy of the form being depicted in the first digital image based, at least in part, on the detected one or more rectangles, and generating, based on the identified duplicate copy of the form, a notification that the first digital image includes at least the first duplicate copy of the form.

Identification of duplicate copies of a form in a document

The present disclosure includes techniques for selecting a candidate presentation style for individual documents for inclusion in an aggregate training data set for a document type that may be used to train an OCR processing engine prior to identifying text in an image of a document of the document type. In one embodiment, text input corresponding to a text sample in a document is received, and an image of the text sample in the document is received. For each of a plurality of candidate presentation styles, an OCR processing engine is trained using a training data set corresponding to the given candidate presentation style, and the OCR processing engine is used, as trained, to identify text in the received image. The OCR processing results for each candidate presentation style are compared to the received text input. A candidate presentation style for the document is selected based on the comparisons.

Aggregate training data set generation for OCR processing

Techniques are disclosed for facilitating optical character recognition (OCR) by identifying one or more regions in an electronic document to perform the OCR. For example a method for identifying information in an electronic document includes obtaining a set of training documents for each template of a plurality of templates for the electronic document, extracting spatial attributes for at least a first label region and at least a first corresponding value region from the set, and training a classifier model based on the extracted spatial attributes, wherein the classifier model is used to identify the information in the electronic document. The spatial attributes represent a position of at least the first label region and at least the first value region within the electronic document.

Performing optical character recognition using spatial information of regions within a structured document

During an information-extraction technique, visual suitability indicators may be displayed to a user of the electronic device to assist the user in acquiring an image of a document that is suitable for subsequent extraction of textual information. For example, an imaging application executed by the electronic device may display, in a window associated with the imaging application, a visual suitability indicator of a tilt orientation of the electronic device relative to a plane of the document. When the tilt orientation falls within a predefined range, the electronic device may modify the visual suitability indicators to provide visual feedback to the user. Then, the electronic device may acquire the image of the document using an imaging device, which is integrated into the electronic device. Next, the electronic device may extract the textual information from the image of the document using optical character recognition.

Growing community of inventors

Mountain View, CA, United States of America

Sreeneel K Maddika