Linkedin Corporation

Volodymyr Zhabiuk

Sanjay Sureshchandra Dubey

Praveen Neppalli Naga

Dhaval Chandrakant Patel

Dhaval H Patel

Jintae Jung

The disclosed embodiments provide a method and system for processing data. During operation, the system obtains a set of records, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics. Next, the system creates a data segment comprising at least one of a forward index and an inverted index for a column in the records. The system then stores the data segment in network-accessible storage and assigns the data segment to a partition. Finally, the system enables querying of the data segment through a query node associated with the partition.

Management of data segments for analytics queries

The disclosed embodiments provide a system that processes data. During operation, the system obtains a set of records, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics. Next, the system creates a data segment from the records by sorting the records by a primary sort column having a first set of possible values, creating a set of dictionaries for a set of columns in the records, and building a set of forward indexes for the columns using the set of dictionaries.

Indexing of data segments to facilitate analytics

The disclosed embodiments provide a system that processes data. During operation, the system obtains a set of records, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics. Next, the system creates, in a data segment comprising the records, an inverted index for a column in the records based on a cardinality of the column. Finally, the system compresses the inverted index based on a jump value associated with record identifiers in the column.

Inverted indexes for accelerating analytics queries

The disclosed embodiments provide a system that processes data. During operation, the system obtains, in a buffer, records to be included in a data segment as the records are generated, wherein each of the records comprises one or more metrics and at least one dimension associated with the one or more metrics. After the buffer reaches a threshold size, the system creates a data segment comprising at least one of a forward index and an inverted index for a column in the records. The system then enables querying of the data segment and a set of offline data segments comprising older records of the one or more metrics and the at least one dimension.

Growing community of inventors

Sunnyvale, CA, United States of America

Volodymyr Zhabiuk