Apple Inc.

Jessica J Peck

Lily Shuting Zhang

Garrett L Weinberg

James Nicholas Jones

Niranjan Manjunath

Willem Mattelaer

Lynn I Streja

Stephen Olivier Lemay

Ieyuki Kawashima

Anumita Biswas

Corey J Peterson

Robert S Bowles Sinclair

Jeffrey L Robbin

Taylor Gerard Carrigan

Kurt W Piersol

Thomas M Alsina

Pedro Mari

William A Sorrentino, Iii

Hugo D Verweij

Harry Joseph Saddler

Timothy Robert Oriol

Benjamin S Phipps

Didier Rene Guzzoni

Mitchell R Lerner

Susan L Booker

Sachin S Kajarekar

Chaitanya Mannemala

Nathan Taylor

Philomena F Lobo

Sophie Lebrecht

Daniel A Castellani

Hiu Yi Chan

Madhu Chinthakunta

Ping Wen Ong

Lewis N Perkins

Luca Simonelli

Cedric Bray

Gavin K Duffy

Stephen G Garcia

Erik Marchi

Harry Simmonds

Helmut Garstenauer

Keith S Brisson

Kumar M Pandya

Stephen H Shum

Matthew R Ward

William A Sorrentino Iii

Lynn Irene Streja

This relates generally to providing search results using a digital assistant based on a displayed application. An example method includes: while displaying a user interface corresponding to an application, receiving, via the one or more input devices, a user input corresponding to a request to initiate a process to perform a search; in accordance with a determination that the user interface includes a search user interface: performing a first search based on the user input using the application; and providing, via the display generation component, a first set of results corresponding to the first search; and in accordance with a determination that the user interface does not include the search user interface: performing a second search based on the user input using a digital assistant of the computer system; and providing, via the display generation component, a second set of results corresponding to the second search.

Providing search results using a digital assistant based on a displayed application

Systems and processes for operating a digital assistant are provided. An example method includes, at an electronic device with one or more processors and memory, while an application is open on the electronic device: receiving a spoken input including a command, determining whether the command matches at least a portion of a metadata associated with an action of the application, and in accordance with a determination that the command matches at least the portion of the metadata associated with the action of the application, associating the command with the action, storing the association of the command with the action for subsequent use with the application by the digital assistant, and executing the action with the application.

Digital assistant control of applications

An example process includes while displaying, on a display, an extended reality (XR) environment: receiving a user input; sampling, with a microphone, a user speech input; in accordance with a determination that the user input satisfies a criterion for initiating a digital assistant, initiating the digital assistant, including: displaying, within the XR environment, a digital assistant indicator at a first location of the XR environment; and while displaying the digital assistant indicator at the first location, providing, by the digital assistant, a response to the user speech input; after providing the response, ceasing to display the digital assistant indicator at the first location; and in accordance with ceasing to display the digital assistant indicator at the first location, displaying the digital assistant indicator at a second location of the XR environment, the second location corresponding to a physical location of an external electronic device implementing a second digital assistant.

Digital assistant interactions in extended reality

Devices and methods for invoking digital assistants are provided. An example method includes displaying, on a display, a virtual reality ('VR') environment, wherein the VR environment includes a first avatar representing a domain-specific digital assistant, and wherein the domain-specific digital assistant corresponds to a first domain; and while displaying the first avatar within the VR environment: receiving a user voice input; determining whether the user voice input represents an intent to invoke the domain-specific digital assistant or an intent to invoke a second digital assistant; and in accordance with a determination that the user voice input represents an intent to invoke the second digital assistant: invoking the second digital assistant; determining, based on the user voice input, a first digital assistant response using the second digital assistant; and outputting, via a speaker, the first digital assistant response.

Devices and methods for invoking digital assistants

Methods and systems for providing a user-targeted report are provided. One method includes receiving an audio signal at a device, encoding a request for the report targeted to the requesting user. A set of incomplete tasks is identified, where tasks have a task-due timestamp and a reported status that indicates whether the task has been associated with a previous report. A set of upcoming tasks is identified from the incomplete tasks. A set of overdue tasks is identified from the incomplete tasks. From upcoming/overdue tasks a set of to-be-reported tasks is generated, via filtering criteria applied to the reported status of each task included in the upcoming/overdue tasks. The report includes a reminder for at least a portion of the tasks included in the set of to-be-reported tasks.

Digital assistant-enabled daily briefings

An example process includes: displaying, on a display of an electronic device, an extended reality (XR) environment corresponding to a copresence session including the electronic device and a second electronic device; while displaying the XR environment: sampling, with a microphone of the electronic device, a first audio input; determining whether the first audio input is intended for a first digital assistant operating on an external electronic device; and in accordance with a determination that the first audio input is intended for the first digital assistant: causing the first digital assistant to provide an audible response to the first audio input, where the audible response is not transmitted to the second electronic device over a shared communication channel for the copresence session.

Digital assistant interactions in copresence sessions

This relates to an intelligent automated assistant in a video communication environment. An example includes, during a video communication session between at least two devices, receiving a voice input at one device, generating and transmitting to a server a textual representation of the voice input, receiving from the server a shared transcription including both the textual representation of the voice input and one or more additional textual representations generated by another device, and determining and presenting one or more candidate tasks based on the shared transcription.

Digital assistant interaction in a video communication session environment

Systems and processes for operating an intelligent automated assistant are provided. In one example process, a spoken user input, including at least a spoken message and an identity of an intended recipient, is received. After storing the spoken message and the recipient identity, if a user becomes proximate to a first electronic device and the identity of the user is determined to match the recipient identity, the spoken message is provided to the user at the first electronic device.

Intelligent message delivery

This relates to an intelligent automated assistant in a video communication session environment. An example method includes, during a video communication session between at least two user devices, and at a first user device: receiving a first user voice input; in accordance with a determination that the first user voice input represents a communal digital assistant request, transmitting a request to provide context information associated with the first user voice input to the first user device; receiving context information associated with the first user voice input; obtaining a first digital assistant response based at least on a portion of the context information received from the second user device and at least a portion of context information associated with the first user voice input that is stored on the first user device; providing the first digital assistant response to the second user device; and outputting the first digital assistant response.

An example process includes: receiving a first natural language input; initiating, by a digital assistant operating on the electronic device, a first task based on the first natural language input; determining whether the first task is of a predetermined type; and in accordance with a determination that the first task is of a predetermined type: determining whether one or more criteria are satisfied; and providing a response to the first natural language input, where providing the response includes: in accordance with a determination that the one or more criteria are not satisfied, outputting a first sound indicative of the initiated first task and a first verbal response indicative of the initiated first task; and in accordance with a determination that the one or more criteria are satisfied, outputting the first sound without outputting the first verbal response.

Sonic responses

Embodiments provide a context-aware digital assistant at multiple user devices participating in a video communication session by using context information from a first user device to determine a digital assistant response at a second user device. In this manner, users participating in the video communication session may interact with the digital assistant during the video communication session as if the digital assistant is another participant in the video communication session. Embodiments further describe automatically determining candidate digital assistant tasks based on a shared transcription of user voice inputs received at user devices participating in a video communication session. In this manner, a digital assistant of a user device participating in a video communication session may proactively determine one or more tasks that a user of the user device may want the digital assistant to perform based on conversations held during the video communication session.

Systems and processes for operating an intelligent automated assistant are provided. An example method includes receiving, from one or more external electronic devices, a plurality of speaker profiles for a plurality of users; receiving a natural language speech input; determining, based on comparing the natural language speech input to the plurality of speaker profiles: a first likelihood that the natural language speech input corresponds to a first user of the plurality of users; and a second likelihood that the natural language speech input corresponds to a second user of the plurality of users; determining whether the first likelihood and the second likelihood are within a first threshold; and in accordance with determining that the first likelihood and the second likelihood are not within the first threshold: providing a response to the natural language speech input, the response being personalized for the first user.

Voice identification in digital assistant systems

A device implementing a system for responding to a voice request includes a processor configured to receive a voice request for content, the voice request corresponding to a first user account, the device being associated with the first user account, a second user account and a default account. The processor if further configured to determine that the content is unavailable via the first user account, and provide, in response to the determining, the content via at least one of the second user account or the default account.

Multi-user devices in a connected home environment

Growing community of inventors

Morgan Hill, CA, United States of America

Jessica J Peck