Apple Inc.

Lynn I Streja

Stephen Olivier Lemay

Jessica J Peck

Ieyuki Kawashima

Garrett L Weinberg

James Nicholas Jones

Pedro Mari

William A Sorrentino, Iii

Harry Joseph Saddler

Didier Rene Guzzoni

Jay Moon

Susan L Booker

Jonathan H Russell

Sreeneel K Maddika

Ari Weinstein

Chaitanya Mannemala

Keith P Avery

Niranjan Manjunath

Michael R Sumner

Myra C Lukens

Cameron Burgess

Lily Shuting Zhang

Karan M Daryanani

Felicia W Edwards

Abhishek Rawat

Michael A Gorbach

Ross A Freeman

Rae L Lasko

Keith C Strickling

An example process includes: while displaying a portion of an extended reality (XR) environment representing a current field of view of a user: detecting a user gaze at a first object displayed in the XR environment, where the first object is persistent in the current field of view of the XR environment; in response to detecting the user gaze at the first object, expanding the first object into a list of objects including a second object representing a digital assistant; detecting a user gaze at the second object; in accordance with detecting the user gaze at the second object, displaying a first animation of the second object indicating that a digital assistant session is initiated; receiving a first audio input from the user; and displaying a second animation of the second object indicating that the digital assistant is actively listening to the user.

Extended reality based digital assistant interactions

An example process includes while displaying, on a display, an extended reality (XR) environment: receiving a user input; sampling, with a microphone, a user speech input; in accordance with a determination that the user input satisfies a criterion for initiating a digital assistant, initiating the digital assistant, including: displaying, within the XR environment, a digital assistant indicator at a first location of the XR environment; and while displaying the digital assistant indicator at the first location, providing, by the digital assistant, a response to the user speech input; after providing the response, ceasing to display the digital assistant indicator at the first location; and in accordance with ceasing to display the digital assistant indicator at the first location, displaying the digital assistant indicator at a second location of the XR environment, the second location corresponding to a physical location of an external electronic device implementing a second digital assistant.

Digital assistant interactions in extended reality

Systems and processes for operating an intelligent automated digital assistant on an electronic device are provided. In particular, this paper describes the automatic registration of voice-activated shortcuts for a particular application at a vocabulary engine associated with the intelligent automated digital assistant. In some embodiments the automatic registration can be performed during installation or update of an application. Automatic registration can also be initiated after entry of particular types of data into the application or a determination that a particular feature of the application is being commonly used.

Voice-activated shortcut registration

Devices and methods for invoking digital assistants are provided. An example method includes displaying, on a display, a virtual reality ('VR') environment, wherein the VR environment includes a first avatar representing a domain-specific digital assistant, and wherein the domain-specific digital assistant corresponds to a first domain; and while displaying the first avatar within the VR environment: receiving a user voice input; determining whether the user voice input represents an intent to invoke the domain-specific digital assistant or an intent to invoke a second digital assistant; and in accordance with a determination that the user voice input represents an intent to invoke the second digital assistant: invoking the second digital assistant; determining, based on the user voice input, a first digital assistant response using the second digital assistant; and outputting, via a speaker, the first digital assistant response.

Devices and methods for invoking digital assistants

An example process includes: displaying, on a display of an electronic device, an extended reality (XR) environment corresponding to a copresence session including the electronic device and a second electronic device; while displaying the XR environment: sampling, with a microphone of the electronic device, a first audio input; determining whether the first audio input is intended for a first digital assistant operating on an external electronic device; and in accordance with a determination that the first audio input is intended for the first digital assistant: causing the first digital assistant to provide an audible response to the first audio input, where the audible response is not transmitted to the second electronic device over a shared communication channel for the copresence session.

Digital assistant interactions in copresence sessions

An example process includes while an electronic device is engaged in a communication session with external device(s): receiving, from a first user of the electronic device, input to invoke a first digital assistant; receiving, from the first user, a natural language input corresponding to a task; in accordance with invoking the first digital assistant, generating, by the first digital assistant, a prompt for further user input about the task; transmitting, to the external device(s), the prompt for further user input about the task; after transmitting the prompt for further user input, receiving, from an external device of the external device(s), a response to the prompt for further user input; initiating, by the first digital assistant, based on the response and information corresponding to the first user stored on the electronic device, the task; and transmitting, to the external device(s), an output indicative of the initiated task.

Digital assistant interaction in a communication session

Growing community of inventors

San Francisco, CA, United States of America

Lynn I Streja