Generating a path to a document element using machine learning

Abstract

Disclosed herein are system, method, and computer program product embodiments for improving web scraping technology by using machine learning to generate parsing expressions. A system receives a request to identify an element in a first document at a target web page. The system downloads and modifies the first document by adding an index value as an attribute to a tag for the element. A query is submitted to a large language model (LLM), including the modified first document, a description of the element, and a request asking the LLM to identify the element based on the description. The system obtains, from the LLM, the index value assigned to the element. The system generates an expression defining a path to the element in the first document using the index returned by the large language model. The system downloads a second document, and parses data of a second element using the expression.