Google Inc.

Oracle International Corporation

Amir Najmi

John Mark Beresniewicz

Yuji Kaneda

Diane L Tang

Marc Berndl

Ankur Taly

Karthik Gopalratnam

Jonathan Fredericks Soule

Dan Liu

Mukund Sundararajan

Rory Abbott Sayres

Jinhau Xu

Adam Jacob Prins

Introduced here are diagnostic platforms able to attribute an output produced by a neural network to its input, as well as communicate the relationship between the output and input in a comprehensible manner. Neural networks are increasingly being used for critical tasks, such as detecting the presence/progression of medical conditions. Accordingly, the importance of explaining how these neural networks produce outputs has grown in importance. By explaining how outputs are produced by a neural network, a diagnostic platform can build trust with medical professionals responsible for interpreting the outputs, identify possible modes of neural network failure, and identify the latent variable(s) responsible for producing a given output.

Attribution methodologies for neural networks designed for computer-aided diagnostic processes

Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for responding to queries. A first user query is received. The first user query is processed including identifying one or more segments in the first user query, a segment representing a word or a phrase. A stand-alone score is determined for each segment of the first user query, wherein the stand-alone score is an indication of a likelihood that the segment represents a stand-alone query and that the segment represents a main topic of the first user query. A historical log of queries is processed to determine query-independent scores for segments that are included in queries represented by the log. The final query-independent scores are used to determine the stand-alone score for each segment of the first query.

Selecting content using query-independent scores of query segments

Technologies relating to determining online advertisement statistics are described. In general, one aspect of the subject matter described in this specification can be embodied in a method that includes receiving a prediction value for a click-through rate for an online advertisement, the prediction value indicating a probability that the online advertisement will be accessed when displayed in an online medium, the prediction value based on a ratio of a number of times the advertisement has been accessed by users to a number of times the advertisement has been displayed to users, identifying at least one condition that is present when the online advertisement is to be displayed, the condition affecting accuracy of the prediction value, and generating a corrected prediction value based on at least one stored shift value associated with the at least one condition.

Determining online advertisement statistics

A computer derives, from a current measurement of an attribute of a system, an instantaneous severity score based on a probability model that is determined from prior measurements. The computer then displays, for several successive moments in time, either the instantaneous severity score or an aggregated severity score obtained from a set of instantaneous severity scores (e.g. by averaging). Each set of scores to be aggregated may be identified by, e.g., a fixed-width time window located at a fixed offset from a moment in time at which the aggregated score is to be displayed. Aggregation suppresses or dampens momentary abnormalities, which would otherwise get displayed. In some embodiments, graphs for multiple categories of metrics are displayed on a single screen, aligned in time, to enable a human operator to form a visual correlation therebetween, e.g. to identify causes of atypical performance in the system.

Graphical display and correlation of severity scores of system metrics

A computer is programmed to fit exponential models to upper percentile subsets of observed measurements for performance metrics collected as attributes of a computer system. The subsets are defined from sets chosen to reduce model bias due to expected variations in system performance, e.g. those resulting from temporal usage patterns induced by end users and/or workload scheduling. Measurement levels corresponding to high cumulative probability, indicative of likely performance anomalies, are extrapolated from the fitted models generated from measurements of lower cumulative probability. These levels are used to establish and to automatically set warning and alert thresholds which signal to (human) administrators when performance anomalies are observed.

Automatic determination of high significance alert thresholds for system performance metrics using an exponentially tailed model

Growing community of inventors

San Francisco, CA, United States of America

Amir Najmi