Apple Inc.

Google Inc.

Patrick M Lister

Jonathan H Russell

Marcel Van Os

Harry Joseph Saddler

Rohit Dasari

Lia T Napolitano

James Edward Stewart

Rajeev Conrad Nongpiur

Chi Yeung Jonathan Ng

Jessica Yuan

Ryan M Orr

Raumi N Sidki

Mark Rajan Malhotra

Jesse William Boettcher

Jeffery Theodore Lee

Jeffrey Alan Boyd

Sophie Le Guen

Prashant Papiahgari Reddy

Josh Buffum

Anthony Michael Fadell

Zachery Webster Kennedy

Michael Mizono

Jared A Crawford

Courtney Dattilo-Green

Systems and processes are disclosed for controlling television user interactions using a virtual assistant. In an example process, a virtual assistant can interact with a television set-top box to control content shown on a television display. Speech input for the virtual assistant can be received from a device with a microphone. The speech input can comprise a query associated with content shown on the television display. A user intent of the query can be determined based on one or more of the content shown on the television display and a viewing history of media content. A result of the query can be caused to be displayed based on the determined user intent.

Intelligent automated assistant for TV user interactions

Various arrangements are detailed herein related to managing video recording based on spoken commands. A system receives a video stream from a video camera and analyzes a field of view in the received video stream to determine a location for one or more identified or potential users. The system can beamform audio from microphones of a home assistant device based on the location of the one or more identified or potential users. The system adjusts an audio output based on the location of the one or more identified or potential users, receives a spoken command from the one or more identified or potential users, and outputs a response to the spoken command.

Video integration with home assistant

Systems and processes are disclosed for operating a digital assistant in a media environment. In an example process, a primary set of media items can be displayed. An audio input containing a media-related request can be received. A primary user intent corresponding to the media-related request can be determined. In accordance with a determination that the primary user intent comprises a user intent to narrow the primary media search query, a second primary media search query corresponding to the primary user intent can be generated. The second primary media search query can be based on the media-related request and the primary media search query. The second primary media search query can be performed to obtain a second primary set of media items. Display of the primary set of media items can be replaced with display of the second primary set of media items.

Intelligent automated assistant for media search and playback

Methods, devices, systems, and means for video integration with a home assistant device are described herein. The home assistant device interacts with a person in a video stream by capturing, using a network-enabled outdoor video camera, a video stream of an outdoor location of a premises at which the person is present and analyzing the person appearing in the captured video stream to determine an identity of the person. Based on determining the identity of the person, the home assistant device announces a presence of the person that is outdoors and outputs instructions to the person.

Various arrangements are detailed herein related to managing video recording. A system can include a network-enabled video camera that captures a video stream of a location and streams the video stream to a cloud-based home assistant integration platform. The system can include a home assistant device comprising a microphone and wireless network interface, the home assistant device configured to receive, via the microphone, a spoken command that requests video, captured by the network-enabled video camera, be recorded. The cloud-based home assistant integration platform may be configured to analyze the video stream captured using the network-enabled video camera to determine an identity of the user. The platform may be further configured to store a portion of the video stream linked to a user account of the user based on the spoken command and the identity of the user.

Various arrangements are detailed herein related to modifying a shopping list using a home assistant device and an integrated video service. A network-enabled video camera may be used to capture a video stream of a location at which a user is present. A home assistant device may receive a spoken command that requests an item be added to a shopping list. The video stream captured using the network-enabled video camera may be analyzed to determine an identity of the user. A shopping list to modify may be determined based on the identity of the user. The determined shopping list may then be modified to indicate the item requested in the spoken command.

Systems and processes are disclosed for real-time updating of virtual assistant media knowledge. Virtual assistant knowledge can be updated with timely information associated with playing media (e.g., a sporting event, a television show, or the like). A data feed can be received that includes data relating events to particular times in a media stream. A user request can be received based on speech input, and the user request can be associated with an event in a media stream or show. In response to receiving the request, the media stream can be cued to commence playback at a time in the media stream associated with the event referred to in the request. In another example, a response to the user request can be generated based on the data relating to the events. The response can then be delivered to the user (e.g., spoken aloud, displayed, etc.).

Real-time digital assistant knowledge updates

Display screen with graphical user interface

Systems and techniques are provided for automatic security system mode selection. A set of signals may be received from sensors distributed in an environment with a security system. The security system may be in a first mode. An occupancy model may be received. An occupancy estimate may be generated for the environment based on the set of signals from the sensors and the occupancy model. Mode rules may be received. The mode rules associate occupancy estimates with modes of the security system. A second mode for the security system may be determined based on the occupancy estimate and mode rules. The second mode may be different from the first mode. The mode of the security system may be automatically changed from the first mode to the second mode.

Automatic security system mode selection

Various arrangements are detailed herein related to managing scheduled events using a home assistant and an integrated video service. Using a network-enabled video camera, a video stream of a location may be captured. The video stream captured may be used to determine an identity of a user present at the location. In response to identifying the identity of the user, access to a cloud-based calendar account for the user may be determined to be authorized. The cloud-based calendar account linked with the user may be accessed. A next calendar event mapped to the cloud-based calendar account that is scheduled to occur within a threshold time duration may be determined. A description linked with the next calendar event may be accessed. A network-enabled smart home assistant device may be used to synthesize speech to announce details related to the next event.

Growing community of inventors

Cupertino, CA, United States of America

Patrick M Lister