International Business Machines Corporation

Manoj Kumar Agarwal

Amitava Kundu

Sriram Kolinjivadi Padmanabhan

Curt Lee Cotner

Prasan Roy

Anindya Neogi

Gautam Kar

Rajesh Sambandhan

Manish Satyapal Gupta

Manish Anand Bhide

Rajeev Gupta

Anca Sailer

Himanshu Gupta

Srinivas Kiran Mittapalli

Mukesh Kumar Mohania

Parviz Kermani

Sanjeev K Gupta

Girish Venkatachaliah

Karen Appleby

Mangesh V Shanbhag

Prasad Manikarao Deshpande

L Venkata Subramaniam

Sumit Negi

Amir Bar-Or

Venkateswara Reddy Madduri

Srilakshmi Kotwal

Rajesh Samabandhan

Oliver Markus Vogel

Sebastian Ertel

Methods, systems, and computer program products for categorizing log records at run-time are provided herein. A computer-implemented method includes generating one or more template signatures to be associated with each of multiple templates, wherein each of the multiple templates comprises a concatenation of one or more words; processing each of multiple log records derived from a data stream to determine a composition of each of the multiple log records; matching one or more of the generated template signatures to each of the multiple log records based on the determined composition of each of the multiple log records; and outputting an identification of (i) each of the multiple log records and (ii) the one or more generated template signatures matched thereto.

Categorizing log records at run-time

Sequences of hierarchical records are aggregated and summarized. A capture log that includes a plurality of operations of a workload is received. A first data structure that models transaction types as sequences of nodes is created. The nodes identify operations in the workload. A present operation and a transaction identifier are read from the capture log. The transaction identifier is dissociated from a first node that identifies a prior operation. The transaction identifier is associated with a second node that identifies the present operation. In a second data structure that associates nodes with transaction identifiers, the first node is dissociated from the transaction identifier and the second node is associated with the transaction identifier. A summary of the workload is generated based, at least in part, on the first and second data structures. The summary includes signatures of transaction types and counts of instances of the transaction types.

Aggregating and summarizing sequences of hierarchical records

Systems and methods for identifying entities sharing a temporal pattern using bipartite graphs are described. In one embodiment, a method includes identifying a temporal pattern in a sequence of bipartite graphs for a sequence of records involving two entity types, where records of the sequence of bipartite graphs vary according to time. An embodiment may color code the edges between entity types in the sequence of bipartite graphs according to the at least one temporal pattern identified (e.g., increasing sales between a business representative and a customer). An embodiment may therefore identify a time-based relationship between at least two entities according to the coded edges. Given the identification of entities having a time-based relationships, groups of these entities may be identified and trends may be derived therefrom (e.g., increasing sales for business units of a particular geographic region).

Systems and methods for discovering temporal patterns in time variant bipartite graphs

Methods and arrangements performing fuzzy search. A contemplated method includes: establishing an edit distance threshold for the fuzzy search; generating an index of items to be searched, via: storing at least one string; and creating substrings corresponding to the at least one string; providing a query string for use in searching; creating substrings corresponding to the query string; comparing substrings of the query string with substrings in the index; designating at least one candidate string based on said comparing; verifying whether each candidate string satisfies the edit distance threshold; and outputting at least one matching string for each candidate string that satisfies the edit distance threshold. Other variants and embodiments are broadly contemplated herein.

Distributed fuzzy search and join with edit distance guarantees

A method, a computer program product and a system identify partition locations within an extended markup language (XML) document without parsing so as to process portions of said document in parallel. The XML document includes sections required to remain continuous. The document is scanned for continuous sections without parsing, and boundaries of the initial partitions are adjusted to reside outside the continuous sections to determine resulting partitions for the document. The resulting partitions may be processed in parallel to provide the document information for storage.

Finding partition boundaries for parallel processing of markup language documents

A system for receiving a declarative specification including a plurality of stages. Each stage specifies an atomic operation, a data input to the atomic operation, and a data output from the atomic operation. The data input is characterized by a data type. Links between at least two of the stages are generated to create a data integration workflow. The data integration workflow is compiled to generate computer code for execution on a parallel processing platform. The computer code configured to perform at least one of data preparation and data analysis.

Declarative specification of data integration workflows for execution on parallel processing platforms

A method for receiving a declarative specification including a plurality of stages. Each stage specifies an atomic operation, a data input to the atomic operation, and a data output from the atomic operation. The data input is characterized by a data type. Links between at least two of the stages are generated to create a data integration workflow. The data integration workflow is compiled to generate computer code for execution on a parallel processing platform. The computer code configured to perform at least one of data preparation and data analysis.

Database operation records are sequentially read from two or more log files. If the transaction identifier is new and the record is not an end-of-transaction record, an open transactions list entry is created. If the transaction identifier is new and the record is an end-of-transaction record, a transaction type list entry is created or updated. If the transaction identifier is not new and is not an end-of-transaction record, an open transactions list entry is updated. If the transaction identifier is not new and the record is an end-of-transaction record, a transaction type list entry is created or updated. When all log file records are read, analytical comparison between the information associated with two or more of the log files in data fields in the transaction type list entries is performed.

Comparing database performance without benchmark workloads

Arrangements and methods for employing empirical evidence to estimate the performance of applications with very few data samples, in complex environments such as dynamic SDP environments, using one or more effective data-plotting models.

System and method for performance management of large scale SDP platforms

Arrangements and methods for employing empirical evidence to estimate the performance of applications with very few data samples, in complex environments such as dynamic SDP environments, using one or more effective, data-plotting models.

Identifying matching transactions. First and second log files contain operation records of transactions in a transaction workload, each file recording a respective execution of the transaction workload, the method comprising. A first record location in the first file and an associated window of a defined number of sequential second record locations in the second file are advanced one record location at a time. Whether each operation record of a complete transaction at a first record location has a matching operation record at one of the record locations in the associated window of second record locations is determined. If so, the complete transaction in the first file and the transaction that includes the matching operation records in the second file are identified as matching transactions.

Matching transactions in multi-level records

A predefined number of matches is identified between records in a first file and records in a second file. For the matches, determine the span of the actual range of record positions in the second file relative to the positions of the operation records in the first file within which all matches were found. If the actual span is smaller than the span of a current defined range of record positions by at least a first threshold value, decrease the span of the current defined range. If the actual span is within a second threshold value of the span of the current defined range, increase the span of the current defined range. If an amount above a third threshold value of operation records in the first file are not matched to operation records in the second file, increasing the span of the current defined range.

Match window size for matching multi-level transactions between log files

Techniques for running an Extract Transform Load (ETL) job in parallel on one or more processors wherein the ETL job comprises use of an extensible markup language (XML) document are provided. The techniques include receiving an XML document input, identifying a node in the XML document at which partitioning of the XML document is to begin, sending partition information to each respective processor, performing a shallow parsing of the XML document in parallel on the one or more processors, wherein each processor performs shallow parsing using the identified partition node until it reaches its identified partition, using the shallow parsing to generate the partition of the input XML document, wherein each processor generates a different partition of the same XML document, and sending each partition in streaming format to an ETL job instance.

Growing community of inventors

Noida, India

Manoj Kumar Agarwal