Google Inc.

Amazon Technologies, Inc.

Amitabh Saikia

Jie Yang

Luis Garcia Pueyo

Mike Bendersky

Marc-Allen Cartright

Vanja Josifovski

MyLinh Yang

Hui Tan

Maureen Heymans

Jinan Lou

Srinidhi Viswanatha

Kashyap Ramesh Puranik

Marc Alexander Najork

Arindam Mandal

Alexander Johannes Smola

Tagyoung Chung

Prakash Krishnan

Raefer Christopher Gabriel

Kamal Tiwari

Randall G Keller

Govindarajan Sundaram Thattai

Amr Ahmed

Venkatesh Manickavasagam

Chien-wei Lin

Jyoti Sood

Devesh Mohan Pandey

Aishwarya Naresh Reganti

Yilin Xiong

Weinan Zhang

Wencong Zhang

Meyyappan Sundaram

Shanchan Wu

Techniques for facilitating natural language interactions with visual interactive content are described. During a build time, a system analyzes various websites and applications relating to a particular user goal to understand website and application navigation and information relating to the user goal. The learned information is used to store configuration data. During runtime, when a user request performance of an action, the system engages in a dialog with the user to complete the user's goal. The system uses the stored configuration data to determine actions to be performed at a website or application to complete the user's goal, and determines system responses to present to the user to facilitate completion of the goal. Such system responses may request information from the user, may inform the user of information displayed at the website or application, etc.

Natural language interactions with interactive visual content

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium determining image relevance responsive to a search query. A method includes, for each resource in a plurality of resources, wherein each resource includes one or more images and text that is separate from each of the one or more images: determining, by a data processing apparatus, and from the text of the resource, resource topics described by the text of the resource. For each of the one or more images, processing the image to determine a set of image topics that describe topics to which content depicted in the image belongs. Determining, by the data process apparatus, one or more topic match scores, wherein each topic match score is a measure of relevance one or more of the image topics of the image to one or more of the resource topics of the resource.

Contextual based image search results

Methods, apparatus, and computer-readable media are provided for analyzing a cluster of communications, such as B2C emails, to generate a template for the cluster that defines transient segments and fixed segments of the cluster of communications. More particularly, methods, apparatus, and computer-readable media are provided for generating and/or applying a trained structured machine learning model for a generated template that can be used to determine, for one or more transient segments of subsequent communications, a corresponding probability that a given semantic label is the correct semantic label for extracted content of the transient segment(s).

Generating and applying a trained structured machine learning model for determining a semantic label for content of a transient segment of a communication

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating and providing animation snippets. In one aspect, a method includes determining page resources that each include a plurality of persistent visual resources, for each of the determined page resources, generating one or more animation snippets from the persistent visual resources, for a page resource for which one or more animation snippets have been generated, selecting one of the animation snippets for display as a search result that identifies the page resource as being responsive to a search query; and providing animation snippet to a user device from which the query was submitted.

Animated snippets for search results

Techniques are described herein for generating and applying event data extraction templates. In various implementations, a data extraction template may be applied to structured communications to extract, from each structured communication, event data associated with a transient markup language path indicated in the data extraction template. The data extraction template may include an event-related semantic data type assigned to the transient markup language path and a strength of association between the transient structural path and the event-related semantic data type. Feedback may be obtained concerning event data extracted from one or more of the structured communications. Based on the feedback, the strength of association between the transient markup language path and the event-related semantic data type may be altered. The data extraction template may then be applied to a subsequent structured communication to extract new event data from the structured communication based on the altered strength of association.

Generating and applying event data extraction templates

Methods, apparatus, and computer-readable media are provided for generating and applying data extraction templates. In various implementations, a corpus of structured communications such as emails may be grouped into clusters based on one or more similarities between the structured communications. A set of structural paths may be identified from structured communications of a particular cluster. One or more structural paths of the set may be classified as transient wherein a count of occurrences of one or more associated segments of text across the particular cluster satisfies a criterion. One or more transient paths may be assigned a semantic data type and/or a confidentiality designation based on various signals. A data extraction template may be generated to extract, from subsequent structured communications, segments of text associated with transient (and in some cases, non-confidential) structural paths.

Generating and applying data extraction templates

Methods, apparatus, systems, and computer-readable media are provided for selecting pattern matching segments suitable for electronic communication clustering. A set of pattern matching segments may be identified that match at least one of a corpus of electronic communication addresses. A measure of coverage of each of the set of pattern matching segments across the corpus of electronic communication addresses may be determined. A score associated with each pattern matching segment may be determined based on the measure of coverage and one or more measures of flexibility associated with each of the set of pattern matching segments. One or more of the pattern matching segments may be selected based on the determine scores. A corpus of electronic communications may then be grouped into a plurality of clusters based on a comparison of the one or more selected pattern matching segments to electronic communication addresses associated with the corpus of electronic communications.

Selecting pattern matching segments for electronic communication clustering

Methods, apparatus, systems, and computer-readable media are provided for generating and applying data extraction templates. In various implementations, a corpus of plain text communications such as emails may be grouped into clusters based on one or more similarities between the plain text communications. One or more segments of communications of a particular cluster may be classified as transient based on textual pattern matching. One or more other segments of the communications of the particular cluster may be classified as transient based on various criteria. One or more transient segments may be assigned a generic and/or specific semantic data type and/or a confidentiality designation based on various signals. A data extraction template may be generated to extract, from subsequent plain text communications, content associated with transient (and in some cases, non-confidential) segments.

Display screen with graphical user interface

Methods, apparatus, systems, and computer-readable media are provided for determining whether communications are attempts at phishing. In various implementations, a potentially-deceptive communication may be matched to one or more templates of a plurality of templates. Each template may represent content shared among a cluster of communications sent by a legitimate entity. In various implementations, it may be determined that an address associated with the communication is not affiliated with one or more legitimate entities associated with the one or more matched templates. In various implementations, the communication may be classified as a phishing attempt based on the determining.

Identifying phishing communications using templates

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for redacting data from a document collection generated for a set of documents that include personal information. The redaction of the data is based in part on a comparison of the document collection to a set of a personal documents of users for which the users have provided explicit approval to use in the processing of the document collection.

Information redaction from document data

Methods and apparatus are described herein for generating and applying event data extraction templates. In various implementations, a set of structural paths may be identified from a corpus of communications. A first structural path of the set of structural paths, associated with a first segment of text, may be classified as transient in response to a determination that a frequency of occurrences of the first segment of text across the corpus satisfies a criterion. Event heuristics may be applied to the communications of the corpus. A determination may be made, based on the applying, that the communications of the corpus are event-related. An event data type may be assigned to the transient structural path based on the applying. An event data extraction template may be generated to extract, from one or more subsequent communications, one or more event-related segments of text associated with the transient structural path.

Methods, apparatus, systems, and computer-readable media are provided for determining whether communications are attempts at phishing. In various implementations, a potentially-deceptive communication may be matched to one or more templates of a plurality of templates. Each template may represent content shared among a cluster of communications sent by a trustworthy entity. In various implementations, it may be determined that an address associated with the communication is not affiliated with one or more trustworthy entities associated with the one or more matched templates. In various implementations, the communication may be classified as a phishing attempt based on the determining.

Growing community of inventors

Mountain View, CA, United States of America

Amitabh Saikia