International Business Machines Corporation

Waterline Data Science, Inc.

Hitachi Vantara Corporation

Business Objects Data Integration, Inc.

Exeros, Inc.

Alexander Gorelik

Lingling Yan

Sriram Kolinjivadi Padmanabhan

James D Spyker

Lev M Tsentsiper

Sachinder S Chawla

Jason Chen

Hon C Thio

Dave Wang

Leon Burda

Salil Datta

Sharad Santhanam

Dongmei Ren

In system for analyzing large data sets, document/file format can be discovered by attempting to parse the file using several parsers to generate a schema, assigning a score to each parsing, and selecting a parser based on the assigned scores. Schema element attributes, such as statistical parameters, can be derived and used in identifying schema elements associated with other files. Attributes of identified schema elements can be used to substitute missing data values with values based on such attributes. Data values corresponding schema elements can be selected and highlighted, and schema elements and/or attributes thereof can be highlighted based on selected data values. From a cluster of files, a lineage relationship between file pairs, indicating whether one file is derived from another, can be determined for several files. In reducing/compacting data, utilization of all available reducers can be optimized according to current utilization of one or more reducers.

Systems and methods for management of data platforms

A computer software implemented method for transforming a first extract transform load (ETL) job having at least some unload transform load (UTL) portions. The method includes the following steps: (i) decomposing the first ETL job into an intermediate set of one or more jobs; and (ii) for each job of the intermediate set, transforming the job into a transactionally equivalent job to yield a final set of one or more jobs. The decomposing is performed so that each job of the intermediate jobs set is a Simple UTL job. The transforming is performed so that each job of the final set includes no UTL portions.

Creation of change-based data integration jobs

According to one embodiment of the present invention, a system determines key relationships between database tables and includes a computer system including at least one processor. The system determines a sampling range for one or more matching columns between first and second database tables. The matching columns satisfy one or more matching criteria and the sampling range is based on quantities of distinct values within the matching columns. Data is sampled from the first and second database tables in accordance with the sampling ranges to determine a sample set. Keys between the first and second database tables are determined based on matching between columns within the sample set. Embodiments of the present invention further include a method and computer program product for determining key relationships between database tables in substantially the same manner described above.

Generating composite key relationships between database objects based on sampling

An apparatus and method are described for the discovery of semantics, relationships and mappings between data in different software applications, databases, files, reports, messages, or systems. In one aspect, semantics and relationships and mappings are identified between a first and a second data source. A binding condition is discovered between portions of data in the first and the second data source. The binding condition is used to discover correlations between portions of data in the first and the second data source. The binding condition and the correlations are used to discover a transformation function between portions of data in the first and the second data source.

Semantic discovery and mapping between data sources

A method, computer program product, and system for analyzing data within one or more databases, comprising selecting one or more databases for analysis, each database comprising one or more database objects comprising one or more data values, applying a function to each data value in each database object within the one or more databases, where the function produces function values limited to a predetermined range, identifying for analysis the data values producing a certain function value within the predetermined range to form a sampled data set, and analyzing the sampled data set to determine relationships between the database objects within and across the one or more databases.

Automatic consistent sampling for data analysis

According to a present invention embodiment, a system determines a relationship between source and target database tables, and includes a computer system including at least one processor. Potential pivot keys of the target database table are determined, and maps are created for each potential pivot key between the database tables based on distinct values. Transformations for each map are generated that enable target data to be produced from source data. The transformations for each potential pivot key are analyzed and the potential pivot key with the transformations that generate the greatest amount of matching data is selected as the resulting pivot key. The database table columns corresponding to the resulting pivot key are determined to be associated by the relationship. Embodiments of the present invention further include a method and computer program product for determining a relationship between source and target database tables in substantially the same manner described above.

Discovering pivot type relationships between database objects

A data source is accessed to provide information. The data source is accessed by defining a plurality of data objects each associated with data within the data source, where the data objects include search information that facilitates searching of the data objects, and the data objects further include display information that pertains to a format in which data obtained from a search of the data objects is displayed, defining one or more relationships linking at least one data object with at least one other data object so as to establish associated data objects with linking relationships, receiving a query to search for data within the data source, retrieving data within the data source satisfying the query in accordance with the search information, where the retrieved data comprises data from at least two associated data objects, organizing and displaying the retrieved data in accordance with the display information, and displaying one or more links associated with the retrieved data so as to enable navigation between associated data objects.

Growing community of inventors

Palo Alto, CA, United States of America

Alexander Gorelik