Yscope Inc.

Uber Technologies, Inc.

Kirk Rodrigues

Yu Luo

Ding Yuan

Ting Chen

Xu Gang Zhao

Michael Stumm

Benjamin Ross

David Lion

Ujwala Prabhakar Tulshigiri

Haiqi Xu

Yupeng Fu

Christopher Peck

Maksym Ivanchenko

Kaibo Wang

Yun Zhang

Systems and methods for compressing and querying data for real-time analytics. The system can receive log data and generate an intermediate representation by parsing a log template and variables from the log data into a columnar format. The method includes generating a compressed intermediate representation associated with an index type and storing the compressed intermediate representation in the columnar format based on the index type. The method includes receiving a search query and analyzing the search query to identify a user defined function. The method includes parsing the search query to convert the search query into one or more predicates that satisfy the search query. The method includes filtering the compressed log data based on the one or more predicates and providing a query result.

Data compression for real-time analytics

Log messages are compressed, searched, and decompressed. A dictionary is used to store non-numeric expressions found in log messages. Both numeric and non-numeric expressions found in log messages are represented by placeholders in a string of log 'type' information. Another dictionary is used to store the log type information. A compressed log message contains a key to the log-type dictionary and a sequence of values that are keys to the non-numeric dictionary and/or numeric values. Searching may be performed by parsing a search query into subqueries that target the dictionaries and/or content of the compressed log messages. A dictionary may reference segments that contain a number of log messages, so that all log message need not be considered for some searches. Compressed logs may be stored and queried on the basis of log type.

Compression, searching, and decompression of log messages

Log messages are compressed, searched, and decompressed. A dictionary is used to store non-numeric expressions found in log messages. Both numeric and non-numeric expressions found in log messages are represented by placeholders in a string of log 'type' information. Another dictionary is used to store the log type information. A compressed log message contains a key to the log-type dictionary and a sequence of values that are keys to the non-numeric dictionary and/or numeric values. Searching may be performed by parsing a search query into subqueries that target the dictionaries and/or content of the compressed log messages. A dictionary may reference segments that contain a number of log messages, so that all log message need not be considered for some searches.

A log of execution of an executable program is obtained. Log messages contained in the log are parsed to generate object identifiers representative of instances of programmatic elements in the executable program. Relationships among the object identifiers are identified. A representation of identified relationships is constructed and outputted as, for example, a visual representation.

Log processing and analysis

Growing community of inventors

Toronto, Canada

Kirk Rodrigues