Amazon Technologies, Inc.

Gary Zhong

Aakarsh Nair

Jonathan Alan Leblang

Collin Charles Davis

Milo Oostergo

Kevin Crews

Yu-Hsiang Cheng

Cassity Barrows Queen

Richard Christopher Green

Sushant Bhatia

Qi Zhou

Inchara Shivalingaiah

This disclosure describes a feedback service that collects feedback for skills, or capabilities, of a virtual assistant that interacts with users, and associates the feedback with the appropriate skills. Virtual assistants interact with users via voice-enabled devices that are backed by voice-processing systems that support various skills of the virtual assistants. Due to large numbers of skills, users are unable to determine which skill is invoked during interactions with virtual assistants, and are thus unable to provide feedback for the skill. The techniques described herein include continuing a speech dialogue with a user after completion of an interaction, and requesting feedback regarding the interaction. Additionally, the techniques may include collecting contextual data for the interaction (e.g., dropped packets, latency caused by jitter, etc.). The feedback service can associate the feedback and contextual data with the particular skill used in the interaction to improve the functioning of the virtual assistant.

Centralized feedback service for performance of virtual assistant

A mapping between environments and devices included therein may be maintained, such that a configuration of each environment is known. Upon detecting that a user is within an environment, and based on a current device state of devices within the environment, an application may be generated and presented to the user via a corresponding user device. The application may allow the user to activate and control the devices within the environment. In particular, the application may depict selectable controls that correspond to functions or operations associated with the different devices within the environment. The application may also be dynamically updated based on an updated current device state of the devices.

Application-based control of devices within an environment

Technologies are disclosed for interacting with a virtual assistant to coordinate, recommend and perform actions. According to some examples, a user may use their voice to interact with a virtual assistant to receive recommendations relating to determining when to perform one or more actions. For example, a user may interact with a virtual assistant to request a recommendation as to when they should leave for the office, leave the office for the day, perform a task, and the like. The recommendation system accesses selected data sources (e.g., calendars, task lists, traffic, transportation schedules, maps, . . . ) to obtain data used in generating the recommendation. In addition, to providing a recommended time, the virtual assistant may also recommend actions to perform. The virtual assistant may also provide notifications to one or more other users that includes information relating to the user leaving.

Interacting with a virtual assistant to coordinate and perform actions

Technologies are disclosed for disambiguating contact information using relationship data using a virtual assistant. A user interacts with a virtual assistant to obtain and utilize contact information. For example, a virtual assistant may allow users to perform an action that utilizes contact information (e.g., make a call to a contact). The virtual assistant utilizes a contact service to identify candidate contacts that are related to the requesting user. The contact service identifies candidate contacts based on relationship data between the requesting user and the stored contacts. For example, the relationship data may indicate that the requesting user is on the same project or team as another contact, that the requesting user has the same role as another contact, that the requesting user is a manager of another contact, as well as other attributes. In some examples, the contact service limits the number of candidate contacts provided to requesting user.

Disambiguating contacts using relationship data

Methods, systems, and computer-readable media for voice-activated selective memory for voice-capturing devices are disclosed. A first voice input from a voice-capturing device is received, via a network, at a service provider environment comprising one or more services. The first voice input comprises one or more utterances from a user of the voice-capturing device. A representation of the first voice input is stored. A second voice input from the voice-capturing device is received, via the network, at the service provider environment. The second voice input represents a command to disregard the first voice input. Based on the second voice input, the representation of the first voice input is deleted.

Voice-activated selective memory for voice-capturing devices

Technologies are disclosed for relaying availability information using a virtual assistant. The virtual assistant can serve as a personal assistant to a user and receive and provide availability information associated with a user. The availability information may include information such as, but not limited to location information about the user, timing information, (e.g., a time the user left), message information (e.g., a message left by the user, a message left by a person looking for the user, . . . ), and the like. The availability information that is provided by the virtual assistant may be different depending on the identity of the requesting user. The virtual assistant may also record messages from requesting users that may be provided to the user upon return and/or while the user is away (e.g., via email, text message, instant message, phone call, and the like).

Relaying availability using a virtual assistant

Techniques for selecting a meeting to join in response to a voice command from a user are described herein. A user may utter a voice command to 'join my meeting' and, in response, the techniques may access a calendar associated with the user to identify the call-in information used to join the meeting. If the calendar indicates that the user is listed as an invitee for multiple meetings at the current time, then the techniques may analyze an array of criteria to determine which meeting the user most likely would like to join.

Selecting meetings based on input requests

Systems and methods are disclosed related to determining device settings for a meeting based in part by parsing text from a meeting invitation or calendar entry, and controlling one or more remotely controllable devices during the meeting according to the determined settings. Based on contextual information determined from a parsed meeting invitation and/or related sources, controllable devices such as remote-controllable window shades, lighting equipment, camera equipment, display screens and/or other in-room equipment may be automatically assigned to determined settings before or during an in-person meeting, conference call or videoconference.

Device control based on parsed meeting information

Systems and methods are disclosed related to determining a likelihood that a user is present in physical proximity to one or more computing devices based on factors such as detecting the user's voice, receiving beaconing signals from a user's mobile device, location information sent from other devices, input received via a camera, and/or other input at various devices that are geographically dispersed. Based on user presence information, a system may determine whether to broadcast a notification for the user to a given device for audible or visual presentation by the device.

Selective notification delivery based on user presence detections

Systems and methods are disclosed related to determining a likelihood that a user is present in physical proximity to one or more computing devices based on factors such as detecting the user's voice, receiving beaconing signals from a user's mobile device, location information sent from other devices, input received via a camera, and/or other input at various devices that are geographically dispersed. Based on user presence information, which may include interaction timestamps, a system may determine confidence levels regarding which of a number of computing devices are in physical proximity of the user at a current time. The confidence scores may be used to determine whether to broadcast a meeting notification or other notification for the user to a given device for audible or visual presentation by the device.

Growing community of inventors

Milpitas, CA, United States of America

Gary Zhong