Microsoft Technology Licensing, LLC

Alexander Tsvetkov

Roy Eisenstadt

Abed El Kader Asi

Royi Ronen

Yarin Kuper

Dean Geckt

Abedelkader Asi

A computer-implemented method is provided that detects jailbreak attempts against generative models, which may involve a shift between benign and malicious content. The method includes determining a probability-based metric for each of a plurality of tokens in a target text using a language model, the probability-based metric being based on a probability at least one preceding token. The probability-based metrics are processed to identify a subset of the plurality of tokens having a change in the probability-based metric with respect to others of the plurality of the tokens not within the subset of the plurality of tokens. A jailbreak attempt in the target text is detected in response to identifying the change in the probability-based metric in the subset of the plurality of tokens.

Detecting jailbreak attempts on generative models

Example solutions for reducing the likelihood of hallucinations by language models, such as large language models (LLMs) are disclosed. By injecting a sufficient range and quantity of curated factual data into a prompt, the likelihood of a hallucination by an LLM may be reduced. This enables language models to be used in a wider range of settings, in which fabrication of facts is problematic, while reducing the need for a human to carefully check the generated text for accuracy. Examples include: generating a summary of a transcript using a summarization model; extracting topic-specific data from stored data using a scoring model; dynamically generating a language model prompt using the topic-specific data and the summary; and generating an output text using a language model and the language model prompt.

Dynamic construction of large language model prompts

The disclosure herein describes interpreting attention-based decisions of summarization outputs generated by a deep learning model. A decision interpretation model obtains attention values defining connections between input tokens associated with a source text and output tokens for a selected portion of a summary associated with the source text. The input tokens having the highest attention values indicating the strongest connections between the input tokens of the source text and an output token of the summary are selected as primary tokens. A semantic similarity between the primary tokens for each attention head and an output token is calculated. The model selects the primary tokens having the closest semantic similarity with the summary portion. A visual cue is generated on or within a portion of the source text corresponding to the primary tokens. The visual cue identifies dominant words in the source text used to explain the summary portion.

Growing community of inventors

Tel Aviv, Israel

Alexander Tsvetkov