Netflix Inc.

Roy Rapoport

Ariel Tseitlin

Jason Chan

Brent Pitman

Brian Harrington

Daniel Muino

Philip Simon Tuffs

Cody Mitchell Rioux

Christopher Sanden

Gregory Burrell

Brian Harington

One embodiment of the invention disclosed herein provides techniques for detecting and remediating an outlier server in a distributed computer system. A control server retrieves a group of time-series data sets associated with a first time period, where each time-series data set represents a performance metric for a different server in a group of servers. The control server generates a cluster that includes two or more of the time-series data sets, where the performance metric for each server that is associated with one of the time-series data sets in the cluster is within a threshold distance from the performance metric for the servers that are associated with the other time-series data sets in the cluster. The control server determines that a particular time-series data set corresponds to a server included in the group of servers and is not included in the cluster, and marks the server as an outlier server.

Server outlier detection

Techniques for adaptive metric collection, metric storage, and alert thresholds are described. In an approach, a metric collector computer processes metrics as a collection of key/value pairs. The key/value pairs represent the dimensionality of the metrics and allows for semantic queries on the metrics based on keys. In an approach, a storage controller computer maintains a storage system with multiple storage tiers ranked by speed of access. The storage computer stores policy data that specifies the rules by which metric records are stored across the multiple storage tiers. Periodically, the storage computer moves database records to higher or lower tiers based on the policy data. In an approach, a metric collector in response to receiving a new metric, generates a predicted metric value based on previously recorded metric values and measures the deviation from the new metric value to determine whether an alert is appropriate.

Adaptive metric collection, storage, and alert thresholds

A security application manages security and reliability of networked applications executing collection of interacting computing elements within a distributed computing architecture. The security application monitors various classes of resources utilized by the collection of nodes within the distributed computing architecture and determine whether utilization of a class of resources is approaching a pre-determined maximum limit. The security application performs a vulnerability scan of a networked application to determine whether the networked application is prone to a risk of intentional or inadvertent breach by an external application. The security application scans a distributed computing architecture for the existence of access control lists (ACLs), and stores ACL configurations and configuration changes in a database. The security application scans a distributed computing architecture for the existence of security certificates, places newly discovered security certificates in a database, and deletes outdated security certificates. Advantageously, security and reliability are improved in a distributed computing architecture.

Method and system for improving security and reliability in a networked application environment

Techniques for evaluating a second version of software. Embodiments selectively route incoming requests to software instances within a plurality of baseline instances and a plurality of canary instances, where the baseline instances run a first software version and the canary instances run the second software version. The software instances are monitored to collect performance data for a plurality of performance metrics. Embodiments calculate aggregate baseline performance metrics, where each of the aggregate baseline performance metrics is calculated based on the collected performance data for the plurality of baseline instances. For each of the performance metrics and canary instances, embodiments calculate a relative performance value that measures the collected performance data for the respective canary instance and for the respective performance metric, relative to the corresponding aggregate baseline performance metric. A final measure of performance is calculated for the second version of software, based on the relative performance values.

Using canary instances for software analysis

Techniques are described for identifying a root cause of a pattern of performance data in a system including a plurality of services. Embodiments provide dependency information for each of the plurality of services, where at least one of the plurality of services is dependent upon a first one of the plurality of services. Each of the plurality of services is monitored to collect performance data for the respective service. Embodiments further analyze the performance data to identify a cluster of services that each follow a pattern of performance data. The first one of the services in the cluster of services is determined to be a root cause of the pattern of performance data, based on the determined dependency information for each of the plurality of services.

Growing community of inventors

Pacifica, CA, United States of America

Roy Rapoport