Apple Inc.

Microsoft Technology Licensing, LLC

Minwoo Jeong

Timothy S Paek

Brandt M Westing

Heriberto Nieto

Stephen Olivier Lemay

Brandon J Newendorp

Srikrishna Sridhar

Shawn R Scully

Charles Maalouf

Joanna Peterson

Shiwen Zhao

Michael Reese Bastian

Roman Holenstein

Ruhi Sarikaya

Keith P Avery

Jamil Dhanani

Young-Bum Kim

Varun Maudgalya

Harveen Kaur

Dilek Zeynep Hakkani-Tur

Christopher Brian Fleizach

Gokhan Tur

Larry Paul Heck

Ye-Yi Wang

Xiaojin Shi

Garrett L Weinberg

Dmytro Rudchenko

Francesco Rossi

Muhammad Amir Shafiq

John M Nefulda

Madhu Chinthakunta

Xinyi Yan

Kenneth S Friedman

Viet Huy Le

Michal K Wegrzynski

Rishabh Yadav

Sommer E Panage

William Caruso

Systems and processes for operating an intelligent automated assistant are provided. An example process includes detecting input representing motion of an electronic device and sampling an audio input with a microphone of the electronic device. The example process further includes determining, based on the audio input and the input representing motion of the electronic device, whether to initiate a virtual assistant session. In accordance with a determination to initiate the virtual assistant session, the example process includes initiating the virtual assistant session. In accordance with a determination not to initiate the virtual assistant session, the example process includes forgoing initiating the virtual assistant session.

Raise to speak

The subject technology receives, from a first sensor of a device, first sensor output of a first type. The subject technology receives, from a second sensor of the device, second sensor output of a second type, the first and second sensors being non-touch sensors. The subject technology provides the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted touch-based gesture based on sensor output of the first type and sensor output of the second type. The subject technology provides a predicted touch-based gesture based on output from the machine learning model. Further, the subject technology adjusts an audio output level of the device based on the predicted gesture, and where the device is an audio output device.

Machine-learning based gesture recognition

The present disclosure generally relates to assigning tasks to various user inputs, and detecting and responding to user inputs. In some embodiments, the present disclosure relates to assigning tasks to various user inputs received on a back surface of a device, and detecting and responding to user inputs on the back surface of the device.

User interfaces for assigning and responding to user inputs

The subject technology receives a neural network model in a model format, the model format including information for a set of layers of the neural network model, each layer of the set of layers including a set of respective operations. The subject technology generates neural network (NN) code from the neural network model, the NN code being in a programming language distinct from the model format, and the NN code comprising a respective memory allocation for each respective layer of the set of layers of the neural network model, where the generating comprises determining the respective memory allocation for each respective layer based at least in part on a resource constraint of a target device. The subject technology compiles the NN code into a binary format. The subject technology generates a package for deploying the compiled NN code on the target device.

Compiling code for a machine learning model for execution on a specialized processor

Structured web pages are accessed and parsed to obtain implicit annotation for natural language understanding tasks. Search queries that hit these structured web pages are automatically mined for information that is used to semantically annotate the queries. The automatically annotated queries may be used for automatically building statistical unsupervised slot filling models without using a semantic annotation guideline. For example, tags that are located on a structured web page that are associated with the search query may be used to annotate the query. The mined search queries may be filtered to create a set of queries that is in a form of a natural language query and/or remove queries that are difficult to parse. A natural language model may be trained using the resulting mined queries. Some queries may be set aside for testing and the model may be adapted using in-domain sentences that are not annotated. The models may be tested using these implicitly annotated natural-language-like queries in an unsupervised fashion.

Exploiting structured content for unsupervised natural language semantic parsing

A computer-implemented technique is described for processing a linguistic item (e.g., a query) in an efficient and scalable manner. The technique interprets the linguistic item using a language understanding (LU) system in a manner that is based on a particular endpoint mechanism from which the linguistic item originated. The LU system may include an endpoint-independent subsystem, an endpoint-dependent subsystem, and a ranking component. The endpoint-independent subsystem interprets the linguistic item in a manner that is independent of the particular endpoint mechanism. The endpoint-dependent subsystem interprets the linguistic item in a manner that is dependent on the particular endpoint mechanism. The ranking component generates final interpretation results based on intermediate results generated by the endpoint-independent subsystem and the endpoint-dependent subsystem, e.g., by identifying the most likely interpretation of the linguistic item.

Scalable endpoint-dependent natural language understanding

Systems and methods for pre-training a sequence tagger with unlabeled data, such as a hidden layered conditional random field model are provided. Additionally, systems and methods for transfer learning are provided. Accordingly, the systems and methods build more accurate, more reliable, and/or more efficient sequence taggers than previously utilized sequence taggers that are not pre-trained with unlabeled data and/or that are not capable of transfer learning/training.

Pre-training and/or transfer learning for sequence taggers

Systems and methods for or training as sequence tagger, such as conditional random field model. More specifically, the systems and methods train a sequence tagger utilizing partially labeled data from crowd-sourced data for a specific application and partially labeled data from search logs. Further, the systems and methods disclosed herein train a sequence tagger utilizing only partially labeled by utilizing a constrained lattice where each input value within the constrained lattice can have multiple candidate tags with confidence scores. Accordingly, the systems and methods provide for a more accurate sequence tagging system, a more reliable sequence tagging system, and a more efficient sequence tagging system in comparison to sequence taggers trained utilizing at least some fully-labeled training data.

Training systems and methods for sequence taggers

Growing community of inventors

Sammamish, WA, United States of America

Minwoo Jeong