Microsoft Technology Licensing, LLC

Ming Li

Marc Todd Friedman

Haibo Lin

Jian H Li

Raghunath Ramakrishnan

Jaliya Nishantha Ekanayake

Zhicheng Yin

Tao Guan

Yongchul Kwon

Jin Sun

José A Blakeley

Ge Wang

Lingwei Li

Jian Li

An anomaly detection system that performs data aggregation against user-defined subsets of multiple variable columns within an aggregation table. Various embodiments enable users to generate rules by defining values for user-defined subsets of variables of an aggregation table that is used to deploy numerous other rules. Thus, if an aggregation table includes hundreds of columns each associated with discrete variables, a user may generate rules for aggregating data into specific variable columns of the aggregation table rather than into all of the hundreds of variable columns. The user may also define a sliding time window for the rule to be applied against and to set an occurrence threshold for the user-defined sliding time window. When a rule is triggered, the anomaly detection system may automatically generate a post-analysis table by preemptively and selectively populating the post-analysis table with values of event data indicated by a cardinality threshold and/or filter parameters.

Enhanced data aggregation techniques for anomaly detection and analysis

Aspects extend to methods, systems, and computer program products for optimally pipelining result sets with fault tolerance in distributed query execution. Distributed computing jobs are optimized by dividing the distributed computing jobs into one or more bubbles for execution. Each bubble can be independently executed, potentially in parallel with other bubbles, when resources to handle the bubble are available. Intra-bubble communication can be streamed between vertices within a bubble. Inter-bubble communication can be stored to durable storage. Bubbles provide a failure boundary for a job graph and re-executing a bubble along with storage of intermediate results in durable storage can be used to recover from failures. When a vertex inside a bubble fails, computation can resume by rescheduling the execution of the failed bubble from the durable inputs for that bubble. Durable storage provides a light-weight failover to handle non-deterministic behavior. Jobs can also leverage streaming to increase performance.

Optimizing pipelining result sets with fault tolerance in distributed query execution

Growing community of inventors

Sammamish, WA, United States of America

Ming Li