Apple Inc.

Meta Platforms, Inc.

Rushin N Shah

Emmanouil Koukoumidis

Benoit F Dumoulin

Rajen Subba

Jinsong Yu

Michael Robert Hanson

Ashish Garg

Heidi Young

Jeremy Gillmor Kahn

Chandrasekhar Iyer

Matthew S Seigel

Devang Kalidas Naik

Harry Joseph Saddler

Matthias Paulik

Robert A Walker

Shweta Grampurohit

Juan Carlos Garcia

Lin Li

Nicholas K Treadgold

Gennaro Frazzingaro

Mubarak Ali Seyed Ibrahim

Blaise Thomson

Gagan Aneja

Justin J Brinegar

Alan Qian

Robert A Walker, Ii

Anuj Kumar

In one embodiment, a method includes receiving a user query inputted on a head-mounted device from the head-mounted device, wherein the user query corresponds to multiple dialog-intents, executing multiple tasks corresponding to the multiple dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the multiple tasks, wherein the stitching model combines the two or more of the execution results based on natural language processing, and wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and sending instructions to the head-mounted device for presenting the multi-perspective response on the head-mounted device.

Generating multi-perspective responses by assistant systems

In one embodiment, a method includes receiving a user query associated with dialog-intents at a client system, executing tasks corresponding to the dialog-intents, generating a multi-perspective response by a stitching model based on two or more of execution results of the tasks, wherein the multi-perspective response comprises a natural-language response combining the two or more execution results, and presenting the multi-perspective response at the client system.

Systems and processes are disclosed for virtual assistant request recognition using live usage data and data relating to future events. User requests that are received but not recognized can be used to generate candidate request templates. A count can be associated with each candidate request template and can be incremented each time a matching candidate request template is received. When a count reaches a threshold level, the corresponding candidate request template can be used to train a virtual assistant to recognize and respond to similar user requests in the future. In addition, data relating to future events can be mined to extract relevant information that can be used to populate both recognized user request templates and candidate user request templates. Populated user request templates (e.g., whole expected utterances) can then be used to recognize user requests and disambiguate user intent as future events become relevant.

Method for extracting salient dialog usage from live data

In one embodiment, a method includes receiving a user query from a client system associated with a first user, determining a plurality of dialog-intents, each being associated with a particular agent of a plurality of agents, executing a plurality of tasks corresponding to the user query via the plurality of agents corresponding to the plurality of dialog-intents, receiving a plurality of execution results corresponding to the plurality of tasks from the plurality of agents, respectively, selecting two or more of the plurality of execution results for combination, generating a multi-perspective response based on the selected execution results by a stitching model, wherein the multi-perspective response comprises a natural-language response combining the selected execution results, and sending instructions for presenting the multi-perspective response to the first user to the client system in response to the user query.

Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors and memory, obtaining a first set of data corresponding to one or more interactions between a user and the digital assistant on the electronic device; obtaining a second set of data corresponding to one or more interactions between the user and an application on the electronic device; and storing the first set of data and the second set of data. The method further includes receiving a set of executable instructions; processing, based on the received set of executable instructions, the first set of data and the second set of data to obtain a third set of data; and transmitting the third set of data to a remote electronic device.

Feedback analysis of a digital assistant

Speech recognition is performed on a received utterance to determine a plurality of candidate text representations of the utterance, including a primary text representation and one or more alternative text representations. Natural language processing is performed on the primary text representation to determine a plurality of candidate actionable intents, including a primary actionable intent and one or more alternative actionable intents. A result is determined based on the primary actionable intent. The result is provided to the user. A recognition correction trigger is detected. In response to detecting the recognition correction trigger, a set of alternative intent affordances and a set of alternative text affordances are concurrently displayed.

User interface for correcting recognition errors

Systems and processes for natural language processing using vocabularies with compressed serialized tries are described in the present disclosure. In one example process, natural language input is received. The natural language input is parsed, using a vocabulary, to determine a corresponding user intent. The parsing includes using a data structure of the vocabulary to map a first word of the natural language input to first semantic information and a second word of the natural language input to second semantic information. The data structure includes pointers that map to a same semantic data object of the vocabulary. The first semantic information and the second semantic information are determined using the same semantic data object. The user intent is determined based on the first semantic information and the second semantic information. Performance of a task corresponding to the determined user intent is initiated.

Natural language understanding using vocabularies with compressed serialized tries

Growing community of inventors

Santa Clara, CA, United States of America

Rushin N Shah