Google Inc.

Robert Stets

Gaurav Bhaya

Justin Lewis

Graeme John Rimmer

Valerie Nygaard

Lewis Jay Hemens

Bogdan Caprita

Simon Michael Rowe

Ken Krieger

Jason Brant Douglas

Richard Dominick Rapp

Raimundo Mirisola

Nicholas Salvatore Arini

Andrew Joseph Alexander Gildfind

Umesh Patil

Saisuresh Krishnakumaran

Ravi Kumar Jain

Mark Jeffrey Foladare

Ruxandra Georgiana Davies

Barnaby John James

Anshul Kothari

Boon-Lock Yeo

Surojit Chatterjee

Martin Brandt Freund

Gaurav Garg

Bo Wang

Alexander Collins

Jonathan William Ray

Momchil Filev

Yunkai Zhou

Bailiang Zhou

Subhadip Sarkar

Venky Ramachandran

Ian James Leader

Timothy Wong O'Connor

Chiu Wah Kelvin So

Vinod Ramachandran Marur

Gangjiang Li

Craig Lawrence Warner

Glenn Wilson

Peng Zhang

Richard Lynn Bennett

Jian Zhou

Ramakrishnan Kandhan

Eric Wood

Paul Feng

Troy L Walker

Lei Zhong

Bradley M Abrams

Wei-Hsin Lee

Jacob D Schonberg

Jianfeng Shen

Terry Van Belle

Vaibhav Vaish

Pramod Adiddam

Nandita Narasimha Prabhu

Yun Huang

David Philip Sisson

Xuemei Gu

Paul Schelte Albert Reitsma

Alexander Ross

Farnaz Azmoodeh

Congxing Cai

A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment is provided. A natural language processor component can parse an input audio signal to identify a request and a trigger keyword. Based on the input audio signal, a direct action application programming interface can generate a first action data structure, and a content selector component can select a content item. An interface management component can identify first and second candidate interfaces, and respective resource utilization values. The interface management component can select, based on the resource utilization values, the first candidate interface to present the content item. The interface management component can provide the first action data structure to the client computing device for rendering as audio output, and can transmit the content item converted for a first modality to deliver the content item for rendering from the selected interface.

Multimodal transmission of packetized data

Modulating packetized audio signals in a voice activated data packet based computer network environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify trigger keyword and request, and generate a first action data structure. The system can identify a content item object based on the trigger keyword, and generate an output signal comprising a first portion corresponding to the first action data structure and a second portion corresponding to the content item object. The system can apply a modulation to the first or second portion of the output signal, and transmit the modulated output signal to the device.

Modulation of packetized audio signals

The present disclosure is generally directed a data processing system for authenticating packetized audio signals in a voice activated computer network environment. The data processing system can improve the efficiency and effectiveness of auditory data packet transmission over one or more computer networks by, for example, disabling malicious transmissions prior to their transmission across the network. The present solution can also improve computational efficiency by disabling remote computer processes possibly affected by or caused by the malicious audio signal transmissions. By disabling the transmission of malicious audio signals, the system can reduce bandwidth utilization by not transmitting the data packets carrying the malicious audio signal across the networks.

Authentication of audio-based input signals

An example method includes receiving, by one or more processors, a representation of an utterance spoken at a computing device; identifying, by a first computational agent from a plurality of computational agents and based on the utterance, a multi-element task to be performed, wherein the plurality of computational agents includes one or more first party computational agents and a plurality of third-party computational agents; and performing, by the first computational agent, a first sub-set of elements of the multi-element task, wherein performing the first sub-set of elements comprises selecting a second computational agent from the plurality of computational agents to perform a second sub-set of elements of the multi-element task.

Multi computational agent performance of tasks

Systems and methods of voice activated thread management in a voice activated data packet based environment are provided. A natural language processor ('NLP') component can receive and parse data packets comprising a first input audio signal to identify a first request and a first trigger keyword. A direct action application programming interface ('API') can generate a first action data structure with a parameter defining a first action. The NLP component can receive and parse a second input audio signal to identify a second request and a second trigger keyword, and can generate a second action data structure with a parameter defining a second action. A pooling component can generate the first and second action data structures into a pooled data structure, and can transmit the pooled data structure to a service provider computing device to cause it device to perform an operation defined by the pooled data structure.

Data structure pooling of voice activated data packets

A selective sensor polling system for a voice activated data packet based computer network environment is provided. A system can receive audio signals detected by a microphone of a device. The system can parse the audio signal to identify trigger keyword and request. The system can select a template for an action data structure with a plurality of fields. The system can determine to poll a first sensor for data for the first field. The system can determine to obtain data in memory previously collected by the second sensor. The system can generate and transmit the action data structure with the data from the sensor and memory, and transmit the action data structure to a third party device.

Selective sensor polling

Systems and methods for automatically determining a content item size may be based on a size of a viewport and a width of a parent element. A script may be configured to determine a size of a viewport, determine a width of a parent element of a resource, and determine a content item size based, at least in part, on the size of the view port and the width of the parent element. A dimension of the determined content item size may be used by a content item selection system to determine a set of content items. A content item selection system may select a content item from the determined set of content items and serve data to effect display of the selected content item in the parent element with the resource.

Resource size-based content item selection

Systems and methods to determine the status of transmitted audio packets. The audio packets can include digital components that are presented by a client computing device. Based on presentation variables and determinations of presentation status, the system can generate a predictive model that can be used to dynamically determines presentation values of audio packets prior to their transmission to a client computing device.

Audio data packet status determination

Growing community of inventors

Mountain View, CA, United States of America

Robert Stets