The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jul. 07, 2026
Filed:
Feb. 22, 2024
Google Llc, Mountain View, CA (US);
Hiroki Furuta, Tokyo, JP;
Ofir Nachum, San Francisco, CA (US);
Kuang-Huei Lee, San Francisco, CA (US);
Izzeddin Gur, San Jose, CA (US);
Shixiang Gu, San Francisco, CA (US);
Google LLC, Mountain View, CA (US);
Abstract
Machine learning models for autonomously navigating the web and/or apps are provided that receive (i) visual input that includes one or more images representing the current, and optionally one or more prior, visual states of the website or app being navigated, and (ii) textual input that represents a user request to be implemented by the model, the current HTML or other code defining the website or app, and optionally additional text like prior model outputs that represent prior steps in the navigation of the website or app by the model. The visual input and textual input are transformed into respective sets of visual and textual tokens, and this multi-modal token set is applied to an encoder. The output of the encoder is applied to a decoder to generate textual output that represents an action to take, relative to the code defining the website or app, to pursue the user request.