International Business Machines Corporation

Rajmohan Chandrahasan

Sameep Mehta

Himanshu Gupta

Manish Anand Bhide

Arvind Agarwal

Pranay Kumar Lohia

Samiulla Zakir Hussain Shaikh

Nitin Kumar Gupta

Andrzej Jan Wrobel

Atul S Kumar

Venkata Nagaraju Pavuluri

Ritwik Chaudhuri

Ankush Gupta

Emma Rose Tucker

Bhanu Mudhireddy

Kirushikesh Db

Embodiments of the present invention provide an approach for exploring interesting data patterns in structured tables through recommending aggregate questions in a conversational data exploration. Specially, interesting features and operators are selected that are used to frame aggregate questions based on user intent and the data. The aggregate questions are ranked based on user persona and interestingness of the questions. The approach dynamically adapts and improves the recommendation of interesting and relevant aggregate questions for the user based on user feedback iteratively.

Recommending aggregate questions in a conversational data exploration

Methods, systems, and computer program products for identifying anomalous transformations using lineage data are provided herein. A computer-implemented method includes generating a set of column profiles for a corresponding set of columns within one or more datasets based at least in part on lineage data and glossary data, wherein the lineage data comprises information related to transformations performed on each column in the set by a computing platform, and wherein the glossary data comprises information related to one or more terms assigned to one or more of the columns; obtaining information related to a new transformation involving at least one column in the set of columns; comparing the new transformation to the set of column profiles to determine whether the new transformation is anomalous; and in response to determining the new transformation is anomalous, outputting an alert to a user of the computing platform.

Identifying anomalous transformations using lineage data

Techniques for automatically determining different data types found in databases are disclosed. In one example, a computer implemented method comprises receiving a portion of identifying information for one or more components of a database, and generating one or more descriptions for the one or more components based at least in part on the portion of the identifying information for the one or more components. The one or more descriptions are inputted to one or more machine learning models, and, using the one or more machine learning models, one or more data types associated with the one or more components are predicted. The prediction is based at least in part on the one or more descriptions.

Automated classification of data types for databases

One embodiment provides a method, including: receiving a target unstructured document for determining whether the target unstructured document comprises biased information; identifying an objective of the target unstructured document by extracting, from the target unstructured document, (i) entities and (ii) relationships between the entities; creating a structured knowledge base, wherein the creating comprises (i) creating an entry in the structured knowledge base corresponding to the target unstructured document, (ii) identifying other unstructured documents having a similarity to the target unstructured document, and (iii) generating an entry in the structured knowledge base corresponding to each of the other unstructured documents; applying a bias detection technique on the structured knowledge base; and providing an indication of whether the target unstructured document comprises bias.

Bias detection for unstructured text

A method, computer system, and a computer program product for generating deep learning model insights using provenance data is provided. Embodiments of the present invention may include collecting provenance data. Embodiments of the present invention may include generating model insights based on the collected provenance data. Embodiments of the present invention may include generating a training model based on the generated model insights. Embodiments of the present invention may include reducing the training model size. Embodiments of the present invention may include creating a final trained model.

Deep learning model insights using provenance data

One embodiment provides a method, including: generating a plurality of ontologies wherein each ontology is generated by: monitoring interactions of a user with lineage information, wherein the monitoring comprises monitoring (i) filter interactions and (ii) access interactions; aggregating the monitored interactions of the user with monitored interactions of other users having a given business role; and generating an ontology for the given business role, wherein the subset comprises (i) event types, (ii) event constraints, (iii) event metadata, and (iv) event context; and upon a user having one of the plurality of business roles accessing lineage information on the data platform, providing a subset of the lineage information.

Generation of lineage data subset based upon business role

A method, computer system, and a computer program product for utilizing provenance data to improve machine learning is provided. Embodiments of the present invention may include collecting provenance data. Embodiments of the present invention may include identifying model quality improvements based on the collected provenance data. Embodiments of the present invention may include identifying related models based on the collected provenance data. Embodiments of the present invention may include recommending model quality improvements to a user.

Growing community of inventors

Kanchipuram, India

Rajmohan Chandrahasan