Intuit, Inc.

Linoy Cohen

Hadas Baumer

Shon Mendelson

Tom Klein

Udi Menkes

Yakov Gazman

Matan Vetzler

An alignment score is generated for a test language model from input data including a number of triplet data structures. The method also includes identifying, responsive to the alignment score failing to satisfy a score threshold, a fail triplet data structure in the number of triplet data structures for which the evaluation score includes the indication of fail. A judge language model is executed on the fail triplet data structure to output a type of misalignment that the test language model produced when the test language model executed on the fail prompt. The judge language model is re-executed on a combination of the fail triplet data structure and the type of misalignment to output a cause of the fail response. An enhanced prompt is generated accordingly and then returned.

Language model improvement through automated prompt engineering

A method of executing a matching language model to perform a many-to-one matching task. The matching task uses a number of tokens that exceeds a token limit of the matching language model. The number of tokens is reduced to a reduced number of tokens by executing a rule-based application on a target entry and a dataset of entries to output a data subset including fewer entries. The reduced number of tokens are reduced within the token limit by executing a sorting language model on the data subset and the target entry to output candidate matching sets. The candidate matching sets are evaluated as to whether a potential match exists with the target entry. The matching language model is executed on the number of candidate matching sets and the target entry to output a selected matching set matching the target entry. The selected matching set is returned.

Large language model ensemble for combinatorial retrieval in one-to-many matching tasks

Growing community of inventors

Petah Tikva, Israel

Linoy Cohen