The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Oct. 25, 2022
Filed:
Sep. 16, 2020
International Business Machines Corporation, Armonk, NY (US);
Md Faisal Mahbub Chowdhury, Woodside, NY (US);
Yu Deng, Yorktown Heights, NY (US);
Alfio Massimiliano Gliozzo, Brooklyn, NY (US);
Ruchi Mahindru, Elmsford, NY (US);
Nandana Mihindukulasooriya, Cambridge, MA (US);
Nicolas Rodolfo Fauceglia, Brooklyn, NY (US);
Gaetano Rossiello, Brooklyn, NY (US);
International Business Machines Corporation, Armonk, NY (US);
Abstract
A method, system, and computer program product for automated evaluation of information retrieval systems are provided. The method accesses a natural language query from a set of natural language queries. The natural language query is associated with a query difficulty level. The method generates one or more natural language responses to the natural language natural language query. Each natural language response is associated with at least one facet of the plurality of facets. The method generates a set of feedback cues. A set of search results for the natural language query are returned. The set of search results include a highest ranked natural language response of the one or more natural language responses. The method generates an evaluation result for the HCIR system for the query difficulty level based on the one or more natural language responses, the set of search results, and the set of feedback cues.