Emc IP Holding Company LLC

Rodolpho Rosa Da Silva

Adriana Bechara Prado

Paulo De Figueiredo Pires

Fábio André Machado Porto

Vitor Silva Sousa

Wagner Dos Santos Vieira

Marcia Lucas Pesce

Altobelli De Brito Mantuan

Techniques are provided for data discovery and data integration in a data lake. One method comprises obtaining data files from a data lake, wherein each data file comprises multiple records having multiple fields; selecting multiple candidate fields from a data file based on a record type; determining a relevance score for each candidate field from the data file based on multiple features extracted from the data file; and clustering the scored candidate fields into clusters of similar domains using a hashing algorithm, wherein a given cluster comprises candidate fields, wherein multiple data files can be integrated based on a domain of the candidate fields in the given cluster. The relevance score for each candidate field is based on multiple features comprising, for example, features that take into account a morphological or semantic similarity between file name, file metadata and/or file records and features that consider statistics of candidate fields in a data file.

Automatic indexing of relevant domains in a data lake for data discovery and integration

Growing community of inventors

Rio de Janeiro, Brazil

Rodolpho Rosa Da Silva