International Business Machines Corporation

Kyle G Christianson

Andrew Ronald Freed

Joshua S Allen

Hassan Nadim

Rafal Przemyslaw Konik

Jim C Chen

Christopher Phipps

Alec J Matschiner

Avery Granum

A method, system and computer-usable medium for detecting headers in various documents, such as PDF and HTML files. The files are converted to a two dimensional array or table, having orthogonal rows and columns. Either rows or columns are determined to include headers. For determining if rows include headers. For each row in the array or table, pair wise comparison is performed for each cell of each column that is orthogonal to that row. The pair wise comparison scores or values are summed up for each orthogonal column to that row and the sum across for all the orthogonal columns to row provide a score or value for that row. Row scores are evaluated relative to one another to determine likelihood of headers in the row. For determining if columns have headers, similar calculation is performed between columns and their orthogonal rows.

Table header detection using global machine learning features from orthogonal rows and columns

A method, system and computer-usable medium are disclosed for finding vertically and horizontally aligned cells in a complex table structure. A file or document, such as an HTML file that defines a complex table includes spanning rows and columns is expanded into a two dimensional (2D) array or table with orthogonal rows and columns, where the spanning rows and columns included cells with copied values or object references. The expanded 2D array or table can be deduplicated row or column wise to determine header alignment of the table.

Intelligent normalization and de-normalization of tables for multiple processing scenarios

A method, computer system, and a computer program product for detecting one or more semantic headers in one or more tabular structures by utilizing a custom pre-trained embeddings model is provided. The present invention may include receiving the custom pre-trained embeddings model. The present invention may also include computing one or more dot product values associated with the one or more tabular structures from the one or more documents based on the context of each cell associated with the one or more tabular structures in the one or more documents. The present invention may then include generating one or more similarity feature values based on the computed one or more dot product values. The present invention may further include detecting the one or more semantic headers associated with the one or more tabular structures based on the one or more similarity feature values.

Semantic header detection using pre-trained embeddings

A method, system and computer-usable medium are disclosed for automated analysis of ground truth using confidence model to prioritize correction options. In certain embodiments, the ground truth data is analyzed to identify review-candidates. A confidence level may be assigned to each of the identified review-candidates and the review-candidates are prioritized, at least in part, using the assigned confidence levels. The review-candidates are electronically presented in prioritized order to solicit verification or correction feedback for updating the ground truth data.

System and method for automated analysis of ground truth using confidence model to prioritize correction options

A method, system and computer-usable medium are disclosed for associating data cells with headers and header labels. In certain embodiments, a table having rows and columns is received, wherein the table includes a plurality of cells, wherein each cell is populated with at least one of a header name, data value, or no information. A determination is made as to whether a cell is a header cell or data cell. If the cell is a header cell, current list of column and current list of row headers are dynamically updated. The current list of column and row headers are assigned to the cell regardless of whether the cell is a header cell or data cell. Headers associated with header cells are used to identify label candidates for the header name of the header cell. The labels may be used to provide additional context for headers within a data cell.

System for extracting header labels for header cells in tables having complex header structures

Techniques for cognitive interpretation of source code are provided. Metadata associated with a section of code in a software project is analyzed to determine a change history of the section of code. A plurality of discussions related to the section of code is evaluated, where each of the plurality of discussions occurred during a code review process. Further, a plurality of support records related to the section of code is analyzed. A sentiment score for the section of code is determined based on the associated metadata, the evaluation of the plurality of discussions, and the analysis of the plurality of support records. Additionally, a display color for the section of code is selected based on the sentiment score. Finally, generation of a graphical user interface (GUI) is facilitated, where the GUI displays the first display color in association with the first section of code.

Interpreting and presenting code based on historical sentiments and interactions

A method, system and computer-usable medium are disclosed for associating data cells with headers and tables having one or more embedded header structures. In certain embodiments, a table having rows and columns is received, wherein the table includes a plurality of cells, wherein each cell is populated with at least one of a header name, data value, or no information, the table having at least one embedded header. A determination is made as to whether a cell is a header cell or data cell. If the cell is a header cell, a count of consecutive column headers is maintained. A current list of column headers is dynamically updated based on the count of the consecutive column headers. Upon encountering a data cell, the current list of column headers is assigned to the data cell.

Growing community of inventors

Rochester, MN, United States of America

Kyle G Christianson