Microsoft Technology Licensing, LLC

Linkedin Corporation

Maneesh Varshney

Srinivas Vemuri

Rui Liu

Khai Tran

Krishna P Puttaswamy Naga

Shrikanth Shankar

Mitul Tiwari

Ya Xu

David J Stein

Walaa Eldin M Moustafa

Vinitha Reddy Gankidi

Sunil Nagaraj

Adwait N Tumbde

Jian Qiao

Carl W Steinbach

Vasanth Rajamani

Suja Viswesan

Wenlei Xie

Harsh Shah

Guillaume B Saint-Jacques

Suvodeep Pyne

Min Shen

Jeremy John Simpson

Ratandeep S Ratti

A unified metrics computation platform decouples user-facing query languages from backend execution engines with the help of an intermediate platform-agnostic language, based on relational algebra procedural query language. The user needs to only specify the computation logic at a single place. By leveraging the intermediate language representation, the unified metrics computation platform can execute the same computation logic on multiple execution engines. The unified metrics computation platform unifies batch, nearline and interactive computations by automatically converting existing user defined batch logic into nearline logic. The user is presented with a unified view of the batch and nearline computation results.

Unified metrics computation platform

The disclosed embodiments provide a system for processing data. During operation, the system organizes fact data to be aggregated into sliding time window features and observation data associated with the fact data into a set of partitions based on a join key. Next, the system sorts the fact data and the observation data within the set of partitions by the join key and timestamps associated with the fact data and the observation data. For each observation record in the observation data, the system aggregates fact records in the sorted fact data that share a value of the join key with the observation record and that fall within a first time window associated with the observation record to produce a sliding time window feature. The system then stores the sliding time window feature in association with the observation record.

Efficient aggregation of sliding time window features

During operation, a system obtains a graph that includes a set of nodes representing users in an online network and a set of edges representing relationships or interactions between pairs of the nodes. Next, the system selects egos containing a subset of the users that are representative of a distribution of an attribute across the users. The system then generates clusters of users, such that each cluster contains an ego and alters selected from the ego's connections in the graph. The system further divides the clusters between a treatment group and a control group, exposes alters in the treatment group to a treatment variant of an A/B test, and exposes alters in the control group to a control variant of the A/B test. Finally, the system outputs a result of the A/B test.

A/B testing using ego network clusters

A computer system and method provide efficient, intuitive, and human-friendly ways of expressing metric-centric queries about multidimensional database data and, at the same, provide a level of flexibility for expressing various metric-centric queries in an interactive manner. The expression techniques help to more easily formulate metric-centric queries that require aggregation operations and joins. The flexible techniques facilitate interactive analysis of multidimensional data from different perspectives including consolidating, drilling down, and slicing and dicing the data. As a result, the analysis task for the data analyst and the programming task for the application developer may be easier, more efficient, and more effective.

Metric-centric transformations of multidimensional database data

The disclosed embodiments provide a system for processing data. During operation, the system obtains a user-defined function (UDF) that is defined using a standard UDF interface for accessing multiple platforms. Next, the system converts data in a platform into a standard form that is supported by the standard UDF interface using a platform-specific wrapper for the UDF and a platform-specific implementation of the standard UDF interface. The system then generates output data in the standard form by applying one or more operations to the data in the standard form. Finally, the system converts the output data into a platform-specific form that is supported by the platform and returns the output data in the platform-specific form.

Platform-independent user-defined functions for data access

This disclosure is directed to large-scale graph processing to determine second-degree connections for members of a social network. A social graph is duplicated into two graphs, where each of the two graphs are partitioned into various partitions. The partitions are each sorted according to a predetermined key selected from each of the graphs. The partitions are then assigned logical Work Units, where a first set of Work Units are determined from a first graph and second set of Work Units are determined from a second graph. The Work Units are determined to be asymmetrical such that the partitions of the first set of Work Units are assigned differently than the partitions of the second set of Work Units. One set of Work Units are loaded in-memory and another set of Work Units are streamed to a mapping module process, which determines the second-degree connections from the sets of Work Units.

Joins and aggregations on massive graphs using large-scale graph processing

A system, method, and apparatus are provided for organizing and joining massive sets of data (e.g., tens or hundreds of millions of event records). A dataset is Blocked by first identifying a partition key, which comprises one or more columns of the data. Each Block will contain all dataset records that have partition key values assigned to that Block. A cost constraint (e.g., a maximum size, a maximum number of records) may also be applied to the Blocks. A Block index is generated to identify all Blocks, their corresponding (sequential) partition key values, and their locations. A second dataset that includes the partition key column(s) and that must be correlated with the first dataset may then be Blocked according to the same ranges of partition key values (but without the cost constraint). Corresponding Blocks of the datasets may then be Joined/Aggregated, and analyzed as necessary.

Organizing, joining, and performing statistical calculations on massive sets of data

Growing community of inventors

San Jose, CA, United States of America

Maneesh Varshney