International Business Machines Corporation

Globalfoundries Inc.

Michael Baessler

Daniel Pittner

Thomas A P Hampp-Bahnmueller

Philipp Hoffmann

Juergen Maletz

Albert Maier

Yannick Saillet

Markus Lorch

Werner Schollenberger

Jochen F Doerre

Peng Hui Jiang

Lars Bremer

Dirk Seider

Mike W Grasselt

Thomas A Hampp-Bahnmueller

Oliver Suhre

Peter Gerstl

Thomas Schwarz

Yan Xu

Jojo Joseph

Pi Jun Jiang

Pavlo Petrenko

Mariya Chkalova

Holger Koenig

Katharina Pfeil

Su Liu

Martin Oberhofer

Si Bin Fan

Jia Zou

Xi Ning Wang

Ling L Zhang

Oliver Koeth

Sebastian Nelke

Ying Shen

Thomas Dürr

Jian F Zhang

Werner Schuetz

Jonathan Roesner

Karin Steckler

Thilo Goetz

Andrea Elias

Several aspects for optimizing unstructured document analysis comprise operating a document system, where the document system comprises a plurality of documents comprising unstructured content and a full-text index; receiving a request to identify documents comprising a type of data elements; selecting a sample out of the plurality of documents; determining data elements of the type in the sample of documents; determining an indicator context expression for the type of data elements out of the determined data elements of the type; determining a query for searching, using a search engine, the full-text index using the indicator context expression; and determining the documents in the document system being compliant to the determined query.

Smart identification of indicator text with full-text search or optimized document analysis

The present disclosure relates to a method of metadata enrichment using an enrichment comprising multiple steps. The method comprises: determining for an input data asset a metadata value descriptive of the input data asset. Characteristics of the metadata value of the input data asset may be determined. At least one informativeness score of the metadata value of the input data asset may be computed using the determined characteristics. An execution of the enrichment step may be skipped in case an input characteristic of the enrichment step is not part of the determined characteristics. In case the input characteristic of the enrichment step is part of the determined characteristics, the enrichment step may be adapted and executed or the enrichment step may be executed without adaptation. Labels resulting from the executed enrichment steps may be combined for providing one or more labels of the data asset.

Optimizing metadata enrichment of data assets

In an approach, a processor receives a request of a document. A processor identifies a set of datasets comprising a sensitive dataset, the set of datasets being interrelated in accordance with a relational model. A processor extracts attribute values of the document. A processor determines that a set of one or more attribute values of the extracted attribute values is in the set of datasets, the set of attribute values being values of a set of attributes. A processor determines that one or more entities of the sensitive dataset can be identified based on relations of the relational model between the set of attributes, where at least part of attribute values of the one or more entities comprises sensitive information. A processor, responsive to determining that the one or more entities can be identified, masks at least part of the set of one or more attribute values in the document.

Protecting sensitive data in documents

Embodiments of the present invention provide methods, computer program products, and systems. Embodiments of the present invention can condense a hierarchy in a data governance system, wherein the hierarchy comprises a root node and at least one child node comprising related sub-trees by determining, for a parent node in the hierarchy of governance system, governance terms and respective assignment relationships from a plurality of information assets, determining usage of the governance term in at least one of a plurality of governance rules, and marking a governance term of the plurality of governance terms for elimination based on the determined assignment relationships and the determined usage of the governance term in the plurality of governance rules. Embodiments of the present invention can then delete the governance term from the hierarchy if the governance term is marked for elimination.

Condensing hierarchies in a governance system based on usage

A database system can comprise records, each record including a set of attributes. The database system can further comprise database views, each database view representing a subset of the set of attributes. Data purpose objects indicating a subset of attributes of the set of attributes and a processing purpose can be stored. Each processing purpose can be associated with one or more entities that authorized access to the subset of attributes of the processing purpose. A request for data for a specific processing purpose and a selected view of the database views can be received. A data purpose object that indicates the specific processing purpose can be retrieved. The subset of attributes represented by the selected view can be compared with the subset of the attributes indicated in the retrieved data purpose object. Values of the subset of attributes of the selected view can be provided.

Conditional access to data

A method for providing one or more random sample documents from a corpus of documents using a search engine is provided. The providing of each of the random sample documents comprises selecting randomly a time window from a set of time windows. A search query is sent to the search engine defining a search for documents of the corpus with time-stamps within the time window defined by the randomly selected time window. In response to the sending of the search query, a search result is receiving from the search engine. The search result comprises a set of the documents of the corpus with time-stamps within the time window. One of the documents comprised by the received set of documents is then selected randomly.

Random sampling from a search engine

The exemplary embodiments disclose a method, a computer program product, and a computer system for protecting sensitive information. The exemplary embodiments may include using an inverted text index for evaluating one or more statistical measures of an index token of the inverted text index, using the one or more statistical measures for selecting a set of candidate tokens, extracting metadata from the inverted text index, associating the set of candidate tokens with respective token metadata, tokenizing at least one document resulting in one or more document tokens, comparing the one or more document tokens with the set of candidate tokens, selecting a set of document tokens to be masked, selecting at least part of the set of document tokens that comprises sensitive information according to the associated token metadata, masking the at least part of the set of document tokens, and providing one or more masked documents.

Masking sensitive information in a document

A system may receive a data glossary comprising a list of terms. The system may then measure a usage dimension for a set of the terms from the list of terms. The system may select a candidate term from the set based on the usage dimension and perform a maintenance action on the candidate terms.

Maintenance of a data glossary

Techniques are described relating to unstructured document processing. An associated computer-implemented method includes identifying a plurality of deduplicated data blocks associated with a collection of unstructured documents. The method further includes sorting the plurality of deduplicated data blocks in descending order based upon at least one block frequency metric, selecting a highest sorted unprocessed deduplicated data block, applying text analytics to the selected deduplicated data block, and applying at least one result of the text analytics to any document among the collection of unstructured documents including the selected deduplicated data block. The method is terminated responsive to satisfaction of at least one stopping condition.

Analyzing deduplicated data blocks associated with unstructured documents

Aspects of the present invention disclose a method, computer program product, and system for governing a set of information assets using an information governance system. The method includes one or more processors applying one or more high-level classification assignment rules to one or more information assets of the set of information assets. Furthermore, the method includes one or more processors applying one or more high-level classification propagation rules to the one or more information assets provided with the high-level classification assignments for propagating the respective high-level classification assignments upwards within a containment hierarchy formed by the set of information assets to one or more superordinate information assets of the set of information assets.

Determining and propagating high level classifications

A method for providing one or more random sample documents from a corpus of documents using a search engine is provided. The providing of each of the random sample documents comprises selecting randomly a time window from a set of time windows. A search query is sent to the search engine defining a search for documents of the corpus with time-stamps within the time window defined by the randomly selected time window. In response to the sending of the search query, a search result is received from the search engine. The search result comprises a set of the documents of the corpus with time-stamps within the time window. One of the documents comprised by the received set of documents is then selected randomly.

A computer implemented method comprising processing the unstructured objects of each record of records of a database for identifying a set of one or more values of attributes in the unstructured objects of the each record. The sets of unstructured attribute values of two records of the database may be compared for determining a similarity level between the two sets. It may be determined whether the two records are representing a same entity based on the comparison result.

Record matching in a database system

In an approach for authentication of a username, a processor maintains a mapping of usernames and realms. A processor receives a username and a time-based one-time password code (TOTP code) for the username based on an authentication application. A processor, upon receiving the TOTP code: determines a realm from the mapping based on the received username and the received TOTP; and requests an entry of a credential relating to the username in the realm. A processor, upon receiving of the requested credential, authenticates the username by determining that the received credential matches an expected credential for the realm.

Second factor based realm selection for federated authentications

In an approach, a processor groups documents into a plurality of groups based on similarity, where: documents of each group have a same document structure; and the document structure is defined by coordinates of text blocks. A processor, for each group of the plurality of groups and for each document of the respective group: retrieves a value of each text block of the respective document in accordance with a document structure of the group; and assigns to each text block of the respective document an attribute that represents the retrieved value of the text block. A processor assigns a first document of the documents to an entity of a database that matches the first document based on the group of text block values and the assigned attributes of the document.

Assigning documents to entities of a database

A computer implemented method is used for sorting data elements of a given set. The method includes performing an evaluation of a first type of usage of each data element. The method includes determining a set of data element candidates dependent on the evaluation of the first type of usage. The method includes performing an evaluation of a second type of usage of each data element of the set of data element candidates. The method includes sorting the data elements of the set of data element candidates dependent on the evaluation of the second type of usage of each data element of the set of data element candidates. The method includes providing the sorted data elements of the set of data element candidates, and in response, receiving a request for a data processing based on the provided sorted data elements of the set of data element candidates.

Growing community of inventors

Neckartenzlingen, Germany

Michael Baessler