Tata Consultancy Services Limited

Pratik Saini

Indrajit Bhattacharya

Tapas Nayak

Subhasish Ghosh

Arpita Kundu

Gautam Shroff

Amit Sangroya

Mrinal Rawat

Chandrasekhar Anantaram

Technical interviewing is important for organizations for assessing a candidate to make hiring decision. For effective technical interviewing, predicting difficulty of long form technical questions is crucial. The present disclosure provides systems and methods for predicting difficulty of long form technical questions using weak supervision from textbooks. Further, zero shot pre-trained large language models and unsupervised template-based technique are used for generating questions. Furthermore, a difficulty score is assigned to the generated questions based on context difficulty and task difficulty. The context difficulty for the generated questions is computed using hierarchical structure of the textbooks, and the task difficulty is computed by determining a similarity between the generated questions and Bloom's taxonomy levels. In the present disclosure, few supervised question difficulty prediction models are trained by means of weak supervision using the generated questions and corresponding difficulty scores and further evaluated for prediction performance using a gold-standard question difficulty dataset.

Methods and systems for predicting difficulty of long form technical questions using weak supervision

The present disclosure a method for document structure based unsupervised long-form technical question generation. Initially, the system receives a textbook document. Further, a PDF metadata is extracted from the textbook document using a Natural Language Processing (NLP) technique. Further, a plurality of structures from the textbook document based on the PDF metadata using an NLP based filtering technique. Further, a plurality of index based question templates and Table of Contents (TOC) based question templates are obtained from a plurality of predefined question templates using the plurality of structures. Further, the generated plurality of long-form technical questions are generated using the obtained index and TOC based question templates. The plurality of long-form technical questions are further evaluated by the system using plurality of metrics. Further, the generated plurality of long-form technical questions are used to finetune a supervised question generation model for generating optimal questions from document structure.

Method and system for document structure based unsupervised long-form technical question generation

For various applications (for example, a Virtual Assistant), mechanisms that are capable of collecting user queries and generating responses are being used. While such systems handle structured queries well, they struggle to or fail to interpret an unstructured Natural Language (NL) query. The disclosure herein generally relates to data processing, and, more particularly, to a method and a system for generating responses to unstructured Natural Language (NL) queries. The system collects at least one NL query as input at a time, and generates a sketch, where the sketch is a structured representation of the unstructured NL query. Further by processing the sketch, the system generates one or more database queries. The one or more database queries are then used to search in one or more associated databases and to retrieve matching results, which are then used to generate response to the at least one NL query.

Method and system for generating a response to an unstructured natural language (NL) query

Growing community of inventors

Noida, India

Pratik Saini