Amazon Technologies, Inc.

Pengkai Zhu

R Manmatha

Peng Tang

Bhavan Ashwinbhai Jasani

Document type specific adapters of a document analysis system are used to provide for additional document types or document specialization when generating answers to user submitted questions targeting information included in a document image provided with the user submitted question. The document analysis system receives a visual question answering (VQA) prompt comprising a document image and a question defining information to be extracted from the document image, generates tokens based on the document image and the question, and adjusts encoding of the tokens using document type specific adapters of a transformer model to extract the information from the document image. A classifier of the document analysis system may determine whether the document image matches a document type supported by the document type specific adapters.

Growing community of inventors

Pasadena, CA, United States of America

Pengkai Zhu