International Business Machines Corporation

Hassan Nadim

Andrew Ronald Freed

Joshua S Allen

Kyle G Christianson

A method, system and computer-usable medium for detecting headers in various documents, such as PDF and HTML files. The files are converted to a two dimensional array or table, having orthogonal rows and columns. Either rows or columns are determined to include headers. For determining if rows include headers. For each row in the array or table, pair wise comparison is performed for each cell of each column that is orthogonal to that row. The pair wise comparison scores or values are summed up for each orthogonal column to that row and the sum across for all the orthogonal columns to row provide a score or value for that row. Row scores are evaluated relative to one another to determine likelihood of headers in the row. For determining if columns have headers, similar calculation is performed between columns and their orthogonal rows.

Table header detection using global machine learning features from orthogonal rows and columns

A method, system and computer-usable medium are disclosed for finding vertically and horizontally aligned cells in a complex table structure. A file or document, such as an HTML file that defines a complex table includes spanning rows and columns is expanded into a two dimensional (2D) array or table with orthogonal rows and columns, where the spanning rows and columns included cells with copied values or object references. The expanded 2D array or table can be deduplicated row or column wise to determine header alignment of the table.

Intelligent normalization and de-normalization of tables for multiple processing scenarios

A method, computer system, and a computer program product for detecting one or more semantic headers in one or more tabular structures by utilizing a custom pre-trained embeddings model is provided. The present invention may include receiving the custom pre-trained embeddings model. The present invention may also include computing one or more dot product values associated with the one or more tabular structures from the one or more documents based on the context of each cell associated with the one or more tabular structures in the one or more documents. The present invention may then include generating one or more similarity feature values based on the computed one or more dot product values. The present invention may further include detecting the one or more semantic headers associated with the one or more tabular structures based on the one or more similarity feature values.

Semantic header detection using pre-trained embeddings

A method, system and computer-usable medium are disclosed for associating data cells with headers and header labels. In certain embodiments, a table having rows and columns is received, wherein the table includes a plurality of cells, wherein each cell is populated with at least one of a header name, data value, or no information. A determination is made as to whether a cell is a header cell or data cell. If the cell is a header cell, current list of column and current list of row headers are dynamically updated. The current list of column and row headers are assigned to the cell regardless of whether the cell is a header cell or data cell. Headers associated with header cells are used to identify label candidates for the header name of the header cell. The labels may be used to provide additional context for headers within a data cell.

System for extracting header labels for header cells in tables having complex header structures

A method, system and computer-usable medium are disclosed for associating data cells with headers and tables having one or more embedded header structures. In certain embodiments, a table having rows and columns is received, wherein the table includes a plurality of cells, wherein each cell is populated with at least one of a header name, data value, or no information, the table having at least one embedded header. A determination is made as to whether a cell is a header cell or data cell. If the cell is a header cell, a count of consecutive column headers is maintained. A current list of column headers is dynamically updated based on the count of the consecutive column headers. Upon encountering a data cell, the current list of column headers is assigned to the data cell.

Growing community of inventors

San Francisco, CA, United States of America

Hassan Nadim