logo

For the Inventor, By the Inventor

The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.

The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.

Patent No.:

US 12387036 B1

PDFFull Text
Date of Patent:
Aug. 12, 2025

Filed:

Oct. 08, 2024

Multimodal agent for efficient image-text interface automation

Applicant:

Anthropic, Pbc, San Francisco, CA (US);

Inventors:

Erich Elsen, San Francisco, CA (US);

Curtis Hawthorne, San Francisco, CA (US);

Augustus Odena, San Francisco, CA (US);

Maxwell Nye, San Francisco, CA (US);

Arushi Somani, San Francisco, CA (US);

Kyle Vigen, San Francisco, CA (US);

Rohan Bavishi, San Francisco, CA (US);

Sagnak Tasirlar, San Francisco, CA (US);

Warut Vijitbenjaronk, San Francisco, CA (US);

Ulas Kirazci, San Francisco, CA (US);

Joe Gershenson, San Francisco, CA (US);

Shaya Zarkesh, San Francisco, CA (US);

Assignee:

Anthropic, PBC, San Francisco, CA (US);

Attorney:

Haynes and Boone LLP

Primary Examiner:

Frank D Mills

Int. Cl.
CPC ...
G06F 40/166 (2020.01); G06F 3/0481 (2022.01); G06F 40/284 (2020.01); G06V 10/77 (2022.01); G06V 10/80 (2022.01); G06V 10/82 (2022.01);
U.S. Cl.
CPC ...
G06F 40/166 (2020.01); G06F 3/0481 (2013.01); G06F 40/284 (2020.01); G06V 10/7715 (2022.01); G06V 10/803 (2022.01); G06V 10/82 (2022.01);
Abstract

A system for image-text agentic interface automation is disclosed. A multimodal agent is configured to process arbitrary-length text sequences and arbitrary-resolution images. A newline insertion logic is configured to interleave a newline character between successive lines of image patches in a plurality of lines of image patches, wherein the newline character specifies an end of a line in an input image. A tokenization logic is configured to translate the input text sequence into a sequence of input text tokens, and to translate the successive lines of image patches interleaved with the newline character into a sequence of input image tokens. A linear projection logic is configured to linearly project a single token stream of the sequence of input text tokens and the sequence of input image tokens into a decoder-only Transformer logic, wherein the linear projection of the single token stream bypasses any embedding lookup.

Find Patent Forward Citations

Loading…