The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Aug. 19, 2025
Filed:
Jul. 13, 2022
Google Llc, Mountain View, CA (US);
Yang Li, Palo Alto, CA (US);
Xin Zhou, Mountain View, CA (US);
Gang Li, Mountain View, CA (US);
Mostafa Dehghani, Amsterdam, NL;
Alexey Alexeevich Gritsenko, Amsterdam, NL;
Google LLC, Mountain View, CA (US);
Abstract
A method includes receiving, via a computing device, a screenshot of a display provided by a graphical user interface of the computing device. The method also includes generating, by an image-structure transformer of a neural network, a representation by fusing a first embedding based on the screenshot and a second embedding based on a layout of virtual objects in the screenshot. The method additionally includes predicting, by the neural network and based on the generated representation, a modeling task output associated with the graphical user interface. The method further includes providing, by the computing device, the predicted modeling task output.