Yscope Inc.

Uber Technologies, Inc.

Yu Luo

Ding Yuan

Kirk Rodrigues

Xu Gang Zhao

Michael Stumm

David Lion

Muhammad Faizanullah

Yongle Zhang

Benjamin Ross

Ujwala Prabhakar Tulshigiri

Haiqi Xu

Yupeng Fu

Christopher Peck

Maksym Ivanchenko

Kaibo Wang

Yun Zhang

Systems and methods for compressing and querying data for real-time analytics. The system can receive log data and generate an intermediate representation by parsing a log template and variables from the log data into a columnar format. The method includes generating a compressed intermediate representation associated with an index type and storing the compressed intermediate representation in the columnar format based on the index type. The method includes receiving a search query and analyzing the search query to identify a user defined function. The method includes parsing the search query to convert the search query into one or more predicates that satisfy the search query. The method includes filtering the compressed log data based on the one or more predicates and providing a query result.

Data compression for real-time analytics

Log messages are compressed, searched, and decompressed. A dictionary is used to store non-numeric expressions found in log messages. Both numeric and non-numeric expressions found in log messages are represented by placeholders in a string of log 'type' information. Another dictionary is used to store the log type information. A compressed log message contains a key to the log-type dictionary and a sequence of values that are keys to the non-numeric dictionary and/or numeric values. Searching may be performed by parsing a search query into subqueries that target the dictionaries and/or content of the compressed log messages. A dictionary may reference segments that contain a number of log messages, so that all log message need not be considered for some searches. Compressed logs may be stored and queried on the basis of log type.

Compression, searching, and decompression of log messages

Log messages are compressed, searched, and decompressed. A dictionary is used to store non-numeric expressions found in log messages. Both numeric and non-numeric expressions found in log messages are represented by placeholders in a string of log 'type' information. Another dictionary is used to store the log type information. A compressed log message contains a key to the log-type dictionary and a sequence of values that are keys to the non-numeric dictionary and/or numeric values. Searching may be performed by parsing a search query into subqueries that target the dictionaries and/or content of the compressed log messages. A dictionary may reference segments that contain a number of log messages, so that all log message need not be considered for some searches.

A log of execution of an executable program is obtained. Log messages contained in the log are parsed to generate object identifiers representative of instances of programmatic elements in the executable program. Relationships among the object identifiers are identified. A representation of identified relationships is constructed and outputted as, for example, a visual representation.

Log processing and analysis

Existing program code, which is executable on one or more computers forming part of a distributed computer system, is analyzed. The analysis identifies log output instructions present in the program code. Log output instructions are those statements or other code that generate log messages related to service requests processed by the program code. A log model is generated using the analysis. The log model is representative of causal relationships among service requests defined by the program code. The log model can then be applied to logs containing log messages generated by execution of the program code, during its normal operation, to group log messages for improved analysis, including visualization, of the performance and behaviour of the distributed computer system.

Systems and processes for computer log analysis

Existing program code, which is executable on one or more computers forming part of a distributed computer system, is analyzed. The analysis identifies log output instructions present in the program code. Log output instructions are those statements or other code that generate log messages related to service requests processed by the program code. A log model is generated using the analysis. The log model is representative of causal relationships among service requests defined by the program code. The log model can then be applied to logs containing log messages generated by execution of the program code, during its normal operation, to group log messages for improved analysis, including visualization, of the performance and behavior of the distributed computer system.

Growing community of inventors

Toronto, Canada

Yu Luo