Google Inc.

Microsoft Technology Licensing, LLC

Tim Phillip Wantland

Brandon Charles Barbello

Donald Allen Barnett

Christopher Max Breithaupt

David Lloyd Jones

Robert Berry

Kaikai Wang

Julian James Odell

Daniel Ramage

Wei Huang

Adarsh Prakash Murthy Kowdle

Christian Paul Charsagua

David Matthew Jones

Michael John Schoenberg

Bryan Woods

Peter Ciccotto

Kyle Pedersen

Seungyeon Kim

Iulia Turc

Alan Ni

John Weber

Lirong Yuan

Anshuman Kumar

Gonzalo Alberto Ramos

Brett Aladdin Barros

Christopher Robert Conover

John Thomas DiMartile, Iii

Alison Lentz

Kristin A Gray

Jonathan Robert Dughi

Bingying Xia

Karen Vertierra

Allison Lee Stanfield

Kristoffer Bernt Henriksson

Matthew Stokes

Melissa Barnhart

Gus Winkleman

Nicholas Aceves

Megan Elizabeth Fazio

Stefano Mazzocchi

John Thomas Dimartile

Systems and methods for determining identifying semantic entities in audio signals are provided. A method can include obtaining, by a computing device comprising one or more processors and one or more memory devices, an audio signal concurrently heard by a user. The method can further include analyzing, by a machine-learned model stored on the computing device, at least a portion of the audio signal in a background of the computing device to determine one or more semantic entities. The method can further include displaying the one or more semantic entities on a display screen of the computing device.

Systems and methods for identifying and providing information about semantic entities in audio signals

Provided are methods, systems, and devices for generating semantic objects and an output based on the detection or recognition of the state of an environment that includes objects. State data, based in part on sensor output, can be received from one or more sensors that detect a state of an environment including objects. Based in part on the state data, semantic objects are generated. The semantic objects can correspond to the objects and include a set of attributes. Based in part on the set of attributes of the semantic objects, one or more operating modes, associated with the semantic objects can be determined. Based in part on the one or more operating modes, object outputs associated with the semantic objects can be generated. The object outputs can include one or more visual indications or one or more audio indications.

Sensor based semantic object generation

A computing system can be configured to input model input that includes context data into a machine-learned model and receive model output that describes one or more semantic entities referenced by the context data. The computing system can be configured to provide data descriptive of the semantic entity or entities to the computer application(s) and receive application output(s) respectively from the computing application(s) in response to providing the data descriptive of semantic entity or entities to the computer application(s). The application output(s) received from each computer application can describe available action(s) of the corresponding computer application with respect to the semantic entity or entities. The computing system can be configured to provide at least one indicator to a user that describes the available action(s) of the corresponding computer applications with respect to the semantic entity or entities.

System and method for selecting and providing available actions from one or more computer applications to a user

The present disclosure is directed to input suggestion. In particular, the methods and systems of the present disclosure can: receive, from a first application executed by one or more computing devices, data indicating information that has been presented by and/or input into the first application; generate, based at least in part on the received data, one or more suggested candidate inputs for a second application executed by the computing device(s); provide, in association with the second application, an interface comprising one or more options to select at least one suggested candidate input of the suggested candidate input(s); and responsive to receiving data indicating a selection of a particular suggested candidate input of the suggested candidate input(s) via the interface, communicate, to the second application, data indicating the particular suggested candidate input.

Methods and systems for input suggestion

A method includes, while a user device is using a first presentation mode to present content to a user, obtaining a current state of the user of the user device. The method also includes, based on the current state of the user, providing, as output from a user interface of the user device, a user-selectable option that when selected causes the user device to use a second presentation mode to present the content to the user. The method further includes, in response to receiving a user input indication indicating selection of the user-selectable option, initiating presentation of the content using the second presentation mode.

Contextual triggering of assistive functions

The methods and systems described herein provide for depth-aware image editing and interactive features. In particular, a computer application may provide image-related features that utilize a combination of a (a) the depth map, and (b) segmentation data to process one or more images, and generate an edited version of the one or more images.

Depth-aware photo editing

Display screen with an animated graphical user interface

Systems are provided to facilitate continuous detection of words, names, phrases, or other sounds of interest and, responsive to such detection, provide a related user experience. The user experience can include providing links to media, web searches, translation services, journaling applications, or other resources based on detected ambient speech or other sounds. To preserve the privacy of those using and/or proximate to such systems, the system refrains from transmitting any information related to the detected sound unless the system receives permission from a user. Such permission can include the user interacting with a provided web search link, media link, or other user interface element.

Trigger sound detection in ambient audio to provide related functionality on a user interface

Growing community of inventors

Mountain View, CA, United States of America

Tim Phillip Wantland