International Business Machines Corporation

Tata Consultancy Services Limited

Kyndryl, Inc.

Indrajit Bhattacharya

Shantanu Ravindra Godbole

Ashish Verma

Kevin W English

Sachindra Joshi

Jeffrey Mark Achtermann

Ashwin Srinivasan

Sangameshwar Suryakant Patil

Tapas Nayak

Ehud Aharoni

Noam Slonim

Girish Keshav Palshikar

Dan Gutfreund

Pratik Saini

Subhasish Ghosh

Prabir Mallick

Soham Datta

Arpita Kundu

Akshit Sharma

Arpan Pal

Balamuralidhar Purushothaman

Mukesh Kumar Mohania

Malolan Chetlur

Ajay Gupta

Vinayaka D Pandit

Chayan Sarkar

Amrita Saha

Tanveer Afzal Faruquie

Yonatan Bilu

Vikas Raykar

Sreyash Kenkre

Pradip Pramanick

Ajay Kattepur

Roy N Bar-Haim

Ullas Balan Nambiar

Ruty Rinott

Vasudeva Varma Kalidindi

Chen Yanover

Daniel Hershcovich

Debayan Mukherjee

Harsimran Bedi

Kumar Avinava Dubey

Jeffrey M Achtermann

Shantanu R Godbole

Atreya Bandyopadhyay

Rajdip Chowdhury

Samiran Pal

Avinash Kumar Singh

Yash Agrawal

Anumita Dasguptabandyopadhyay

Godbole Shantanu

Francesco Dinuzzo

Technical interviewing is important for organizations for assessing a candidate to make hiring decision. For effective technical interviewing, predicting difficulty of long form technical questions is crucial. The present disclosure provides systems and methods for predicting difficulty of long form technical questions using weak supervision from textbooks. Further, zero shot pre-trained large language models and unsupervised template-based technique are used for generating questions. Furthermore, a difficulty score is assigned to the generated questions based on context difficulty and task difficulty. The context difficulty for the generated questions is computed using hierarchical structure of the textbooks, and the task difficulty is computed by determining a similarity between the generated questions and Bloom's taxonomy levels. In the present disclosure, few supervised question difficulty prediction models are trained by means of weak supervision using the generated questions and corresponding difficulty scores and further evaluated for prediction performance using a gold-standard question difficulty dataset.

Methods and systems for predicting difficulty of long form technical questions using weak supervision

The present disclosure a method for document structure based unsupervised long-form technical question generation. Initially, the system receives a textbook document. Further, a PDF metadata is extracted from the textbook document using a Natural Language Processing (NLP) technique. Further, a plurality of structures from the textbook document based on the PDF metadata using an NLP based filtering technique. Further, a plurality of index based question templates and Table of Contents (TOC) based question templates are obtained from a plurality of predefined question templates using the plurality of structures. Further, the generated plurality of long-form technical questions are generated using the obtained index and TOC based question templates. The plurality of long-form technical questions are further evaluated by the system using plurality of metrics. Further, the generated plurality of long-form technical questions are used to finetune a supervised question generation model for generating optimal questions from document structure.

Method and system for document structure based unsupervised long-form technical question generation

The method and system of the present disclosure facilitate automatic, and in an objective manner, selection of an optimal set of technical questions, from a question bank, personalized for a candidate. This ensures consistent, standardized, efficient and objective interviews that result in high quality recruitment, given the diversity of candidates, complexity of job requirements and interviews are inherently subjective. The state-of-the-art depends on responses so far, to generate on-the-fly questions causing a cognitive load on the interviewer. Also, there is no guarantee on the breadth and depth of concepts assessed in each interview. In the present disclosure, skill graphs are employed to create a semantically rich and detailed characterization of questions in terms of concepts. Optimization formulation uses the skill graph to generate constraints, content balancing and objective functions for selection of questions.

Integer linear programming based generation of a personalized optimal questionnaire using a skill graph

Questions play a central role in assessment of a candidate's expertise during an interview or examination. However, generating such questions from input text documents manually needs specialized expertise and experience. Further, techniques that are available for automated question generation require input sentence as well as an answer phrase in that sentence to generate question. This in-turn requires large training datasets consisting tuples of input sentence answer-phrase and the corresponding question. Additionally, training datasets are available are for general purpose text, but not for technical text. Present application provides systems and methods for generating technical questions from technical documents. The system extracts meta information and linguistic information of text data present in technical documents. The system then identifies relationships that exist in provided text data. The system further creates one or more graphs based on the identified relationships. The created graphs are the used by the system to generate technical questions.

Automated systems and methods for generating technical questions from technical documents

This disclosure relates generally to long-form answer extraction and, more particularly, to long-form answer extraction based on combination of sentence index generation techniques. Existing answer extractions techniques have achieved significant progress for extractive short answers; however, less progress has been made for long form questions that require explanations. Further the state-of-art long-answer extractions techniques result in poorer long-form answers or not address sparsity which becomes an issue longer contexts. Additionally, pre-trained generative sequence-to-sequence models are gaining popularity for factoid answer extraction tasks. Hence the disclosure proposes a long-form answer extraction based on several steps including training a set of generative sequence-to-sequence models comprising a sentence indices generation model and a sentence index spans generation. The trained set of generative sequence-to-sequence models is further utilized for model long-form answer extraction based on a union of several sentence index generation techniques comprising a sentence indices and a sentence index spans.

Method and system for long-form answer extraction based on combination of sentence index generation techniques

This disclosure relates generally to generating annotations and field-names for a relational schema. Typically, most domains have relational database (RDB) system built for them instead of domain ontologies and usually linguistic information of the schema is not used to recover the domain terms. The disclosed method and system facilitate generating annotations and field-names for a relational schema, while considering the linguistic information of a schema by using a trained model, trained through a proposed training technique. The trained model comprises of at least one knowledge graph and a set of associated parameters. The trained model is further used to perform a plurality of tasks, wherein the plurality of tasks include generating a plurality of new fieldnames for a relational schema through a stochastic generative process and for generating a new annotation for a fieldname of a relational schema through a probabilistic inference technique.

Method and system for generating annotations and field-names for relational schema

A method and system. Target clusterability is calculated as an average of a respective clusterability of at least one target data item comprised by a target domain. Target-side matchability is calculated as an average of a respective matchability of each target centroid of the target domain to source centroids of a source domain, wherein the source domain comprises at least one source data item. Source-side matchability is calculated as an average of a respective matchability of each source centroid of said source centroids to the target centroids. Source-target pair matchability is calculated as an average of the target-side matchability and the source-side matchability. Cross-domain clusterability between the target domain and the source domain is calculated as a linear combination of the calculated target clusterability and the calculated source-target pair matchability. The cross-domain clusterability is transferred to a device.

Cross-domain clusterability evaluation for cross-guided data clustering based on alignment between data domains

This disclosure relates generally to human-robot interaction (HRI) to enable a robot to execute tasks that are conveyed in a natural language. The state-of-the-art is unable to capture human intent, implicit assumptions and ambiguities present in the natural language to enable effective robotic task identification. The present disclosure provides accurate task identification using classifiers trained to understand linguistic and semantic variations. A mixed-initiative dialogue is employed to resolve ambiguities and address the dynamic nature of a typical conversation. In accordance with the present disclosure, the dialogues are minimal and directed to the goal to ensure human experience is not degraded. The method of the present disclosure is also implemented in a context sensitive manner to make the task identification effective.

Conversational systems and methods for robotic task identification using natural language

One embodiment provides a method, including: receiving input identifying a goal of a student, wherein the goal indicates (i) a target concept to be learned by the student and (ii) a desired expertise corresponding to the target concept; receiving input indicating constraints comprising (i) a time budget and (ii) an effort budget; and generating a syllabus for the student to reach the goal, wherein the syllabus comprises a sequence of sub-concepts to be learned for reaching the goal, by: producing a plurality of alternative sequences of sub-concepts for reaching the identified goal, each sequence of sub-concepts having both a determined, corresponding (i) effort cost and (ii) time cost, wherein determining the corresponding effort cost and time cost comprises identifying relationships between sub-concepts; and determining, from the plurality of alternative sequences, a particular one of the sequences that reaches the target concept at the desired expertise and fulfills the indicated constraints.

Personalized syllabus generation using sub-concept sequences

A method comprising using at least one hardware processor for: identifying relations between pairs of claims of a set of claims; aggregating the claims of the set of claims into a plurality of clusters based on the identified relations; generating a plurality of arguments from the plurality of clusters, wherein each of the arguments is generated from a cluster of the plurality of clusters, and wherein each of the arguments comprises at least one claim of the set of claims, scoring each possible set of a predefined number of arguments of the plurality of arguments, based on a quality of each argument of the predefined number of arguments and on diversity between the predefined number of arguments; and generating a speech, wherein the speech comprises a top scoring possible set of the possible set of the predefined number of arguments.

Automatic generation of a speech by processing raw claims to a set of arguments

A method comprising using at least one hardware processor for: receiving (a) a proposition and (b) a plurality of claims; identifying a local claim polarity of each claim of the plurality of claims with respect to the proposition; calculating a pairwise claim polarity agreement score for each pair of claims of the pairs of claims reflecting the likelihood of said each pair of claims to have the same claim polarity, wherein the pairwise claim polarity agreement score is associated with each claim of the pair of claims; and determining a global claim polarity for each claim of the plurality of claims based on the local claim polarity of the claim and pairwise claim polarity agreement scores associated with said each claim.

Claim polarity identification

A method and system for evaluating cross-domain clusterability upon a target domain and a source domain. Target clusterability is calculated as an average of a respective clusterability of at least one target data item comprised by the target domain. Target-side matchability is calculated as an average of a respective matchability of each target centroid of the target domain to source centroids of the source domain, wherein the source domain comprises at least one source data item. Source-side matchability is calculated as an average of a respective matchability of each source centroid of said source centroids to the target centroids. Source-target pair matchability is calculated as an average of the target-side matchability and the source-side matchability. Cross-domain clusterability between the target domain and the source domain is calculated as a linear combination of the calculated target clusterability and the calculated source-target pair matchability. The cross-domain clusterability is transferred to a device.

Systems and associated methods for enhanced concept understanding in large document collections through phrase clustering are described. Embodiments take as input an initial set of phrases and estimate centroids using a clustering process. Embodiments then generate new phrases around each of the current centroids using the current phrases. These new phrases are added to the current set, and the clustering process is iterated. Upon convergence, embodiments finalize clusters based on phrases of any given length.

Phrase clustering

Determining synonyms of words in a set of documents. Particularly, when provided with a word or phrase as input, in exemplary embodiments there is afforded the return of a predetermined number of 'top' synonym words (or phrases) for an input word (or phrase) in a specific collection of text documents. Further, a user is able to provide ongoing and iterative positive or negative feedback on the returned synonym words, by manually accepting or rejecting such words as the process is underway.

Method and system for interactively finding synonyms using positive and negative feedback

Growing community of inventors

New Delhi, India

Indrajit Bhattacharya