Apple Inc.

Niranjan Manjunath

Avi Bar-Zeev

Rahul Nair

Golnaz Abdollahian

Jessica J Peck

Willem Mattelaer

Lily Shuting Zhang

Jonathan H Russell

Xinyuan Huang

Garrett L Weinberg

William W Luciw

Lynn I Streja

Scott M Andrus

Felicia W Edwards

Rae L Lasko

Keith C Strickling

An example process includes while an electronic device is engaged in a communication session with external device(s): receiving, from a first user of the electronic device, input to invoke a first digital assistant; receiving, from the first user, a natural language input corresponding to a task; in accordance with invoking the first digital assistant, generating, by the first digital assistant, a prompt for further user input about the task; transmitting, to the external device(s), the prompt for further user input about the task; after transmitting the prompt for further user input, receiving, from an external device of the external device(s), a response to the prompt for further user input; initiating, by the first digital assistant, based on the response and information corresponding to the first user stored on the electronic device, the task; and transmitting, to the external device(s), an output indicative of the initiated task.

Digital assistant interaction in a communication session

In an exemplary technique, audio information responsive to received input is provided. While providing the audio information, one or more conditions for stopping the provision of audio information are detected, and in response, the provision of the audio information is stopped. After stopping the provision of the audio information, if the one or more conditions for stopping the provision of audio information have ceased, then resumed audio information is provided, where the resumed audio information includes a rephrased version of a previously provided segment of the audio information.

Providing audio information with a digital assistant

This relates to an intelligent automated assistant in a video communication environment. An example includes, during a video communication session between at least two devices, receiving a voice input at one device, generating and transmitting to a server a textual representation of the voice input, receiving from the server a shared transcription including both the textual representation of the voice input and one or more additional textual representations generated by another device, and determining and presenting one or more candidate tasks based on the shared transcription.

Digital assistant interaction in a video communication session environment

In an exemplary technique, speech input including one or more instructions is received. After the speech input has stopped, if it is determined that one or more visual characteristics indicate that further speech input is not expected, a response to the one or more instructions is provided. If it is determined that one or more visual characteristics indicate that further speech input is expected, a response to the one or more instructions is not provided.

This relates to an intelligent automated assistant in a video communication session environment. An example method includes, during a video communication session between at least two user devices, and at a first user device: receiving a first user voice input; in accordance with a determination that the first user voice input represents a communal digital assistant request, transmitting a request to provide context information associated with the first user voice input to the first user device; receiving context information associated with the first user voice input; obtaining a first digital assistant response based at least on a portion of the context information received from the second user device and at least a portion of context information associated with the first user voice input that is stored on the first user device; providing the first digital assistant response to the second user device; and outputting the first digital assistant response.

Embodiments provide a context-aware digital assistant at multiple user devices participating in a video communication session by using context information from a first user device to determine a digital assistant response at a second user device. In this manner, users participating in the video communication session may interact with the digital assistant during the video communication session as if the digital assistant is another participant in the video communication session. Embodiments further describe automatically determining candidate digital assistant tasks based on a shared transcription of user voice inputs received at user devices participating in a video communication session. In this manner, a digital assistant of a user device participating in a video communication session may proactively determine one or more tasks that a user of the user device may want the digital assistant to perform based on conversations held during the video communication session.

In an exemplary technique for providing audio information, an input is received, and audio information responsive to the received input is provided using a speaker. While providing the audio information, an external sound is detected. If it is determined that the external sound is a communication of a first type, then the provision of the audio information is stopped. If it is determined that the external sound is a communication of a second type, then the provision of the audio information continues.

The present disclosure relates to resolving natural language ambiguities with respect to a simulated reality setting. In an exemplary embodiment, a simulated reality setting having one or more virtual objects is displayed. A stream of gaze events is generated from the simulated reality setting and a stream of gaze data. A speech input is received within a time period and a domain is determined based on a text representation of the speech input. Based on the time period and a plurality of event times for the stream of gaze events, one or more gaze events are identified from the stream of gaze events. The identified one or more gaze events is used to determine a parameter value for an unresolved parameter of the domain. A set of tasks representing a user intent for the speech input is determined based on the parameter value and the set of tasks is performed.

Growing community of inventors

San Jose, CA, United States of America

Niranjan Manjunath