International Business Machines Corporation

Shashank Mujumdar

Nitin Kumar Gupta

Sameep Mehta

Hima Patel

Ruhi Sharma Mittal

Krishnasuri Narayanam

Rahul Jain

Manjit Singh Sodhi

Arvind Agarwal

Deepak Vijaykeerthy

Vitobha Munigala

Prerna Agarwal

Abhishek Mitra

A method, computer program, and computer system are provided for collecting and annotating data based on user preference. Unlabeled data corresponding to one or more entries within a dataset is received. Pseudo-labeled data is generated based on the unlabeled data. Based on one or more quality metrics, each entry from among the pseudo-labeled data is determining to be included within a final dataset. A user is prompted for annotations corresponding to entries of the pseudo-labeled data included within the final dataset. A determination is made as to whether additional data is needed based on comparing the final dataset to the one or more quality metrics, and the additional information is collected if the final dataset does not meet the quality metrics.

Annotating and collecting data-centric AI quality metrics considering user preferences

Processing within a computing environment is facilitated using a corpus processing system to assess and enhance quality of a corpus of unstructured documents for a specified task. The processing includes referencing, by a corpus processing engine, the corpus of unstructured documents to obtain unstructured document data, and applying, by a corpus quality metrics engine, a set of quality metrics to the document data to obtain a set of quality metric scores. Further, the process includes automatically selecting, by a quality metric selection engine, a subset of task-relevant quality metrics using the quality metric scores and the specified task, and automatically transforming, at least in part, multiple documents of the corpus to remediate one or more identified issues with the documents. The automatically transforming results in remediated documents tuned for the specified task, which are provided for the specified task to be performed.

Corpus quality processing for a specified task

Automatically selecting data for machine learning datasets is provided. The method comprises receiving an input dataset and user-specified data quality metrics. The input dataset is matched to a subset of candidate datasets in a repository according to schema characteristics. A second subset of candidate datasets having a distance from the input dataset above a specified threshold is selected from the first subset of candidate datasets. The second subset of candidate datasets are merged into a merged dataset. Top ranked samples above a specified second threshold are identified from the merged dataset based on the user-specified data quality metrics. The input dataset, augmented with the top ranked samples, is returned to the user.

Automatically selecting relevant data based on user specified data and machine learning characteristics for data integration

A system may include a memory and a processor in communication with the memory. The processor may be configured to perform operations. The operations may include extracting a first multiple dimensional dataset and a second multiple dimensional dataset and encoding the first multiple dimensional dataset into a first encoded dataset and the second multiple dimensional dataset into a second encoded dataset. The operations may include combining the first encoded dataset and the second encoded dataset into a combined encoded dataset and conveying the combined encoded dataset to a user.

Optimized transmission and consumption of digital content

A method, a system, and a computer program product are provided for analyzing an instructional video. Video data of an instructional video is analyzed to form multiple units of work. Each unit of work is a respective grouping of video frames of the instructional video based on a respective logical combination of activities associated therewith. Each unit of work is analyzed to produce a respective action graph of activities included in the unit of work, the respective action graph indicating interdependencies among the activities included therein. Interdependencies among activities across the units of work are determined to form a critical path graph. A received query is processed to provide troubleshooting assistance with respect to the instructional video based on the units of work, the action graphs, the critical path graph, and a knowledge base including information related to a subject matter of the instructional video.

Video processing for troubleshooting assistance

One embodiment provides a method, including: receiving a query from a user requesting assistance regarding instructions for performing a task; identifying, within steps of the instructions, words that can be visualized, wherein the identifying comprises identifying relationships between terms within the query to generate a step query; retrieving, for each of the steps, a plurality of images representing the identified words; identifying at least one object occurring within the plurality of images corresponding to more than one of the steps; selecting an image for each of the steps of the instructions, wherein the selecting an image comprises selecting an image for each step such that the identified at least one object is represented similarly in each selected image including the identified at least one object; and presenting the instructions as visualized instructions by presenting the selected images for each of the steps in order.

Cohesive visualized instruction representation

One embodiment provides a method, including: receiving, from a client, (i) a task of annotating information, (ii) a set of instructions for performing the task, and (iii) client annotations for a subset of the information within the task; assigning the subset to a plurality of annotators; obtaining (i) annotator annotations for the subset and (ii) a response time for providing the annotator annotation for each piece of information within the subset; identifying improvements to the set of instructions by (i) comparing the annotator annotations to the client annotations and (ii) identifying discrepancies made by the annotators in view of the response time; and generating a new set of instructions, wherein the generating comprises (i) identifying at least one feature of the information that distinguishes correctly annotated information from incorrectly annotated information and (ii) generating an instruction from the at least one feature.

Annotation task instruction generation

One embodiment provides a method, including: receiving, from an application, information for storage in a storage management system, wherein the storage management system comprises a plurality of storage layers, each storage layer having a different performance and a different cost than the other storage layers; labeling the information with one of a plurality of labels relevant to the application; assigning the information a performance tolerance value based upon the label of the information, wherein the performance tolerance value comprises an estimate of the performance requirement required by the application storing the information; determining a storage layer for storage of the information, wherein the determining comprises identifying one of the plurality of storage layers corresponding to the label of the information and updating metadata of a logical volume corresponding to the information with the performance tolerance value; and sending the information to the determined storage layer.

Growing community of inventors

Nagpur, India

Shashank Mujumdar