Intuit, Inc.

Aleksandr Kim

Yair Horesh

Itay Margolin

Certain aspects of the disclosure provide a method for detecting data collection errors by processing error data with a plurality of regression models to generate a plurality of predicted error rates over a plurality of time intervals. The method includes determining an error mode by applying a set of policy rules optimized for determining the error mode to the plurality of predicted error rates.

Real-time remote system shutdown prediction

A processor may identify a plurality of data sets subject to upcoming update processing in a next update cycle. For each of the plurality of data sets, the processor may determine a probability that data included in the data set has changed since a most recent update processing. The processor may exclude a first subset of the plurality of data sets having respective probabilities below a threshold value from the upcoming update processing until the respective probabilities are determined again in a subsequent update cycle. The processor may perform the upcoming update processing on the plurality of the data sets not included in the first subset, where the upcoming update processing may include obtaining updated data from at least one external data source.

Efficient automatic web scraping systems and methods

A method and system that proactively generate alerts for updating a scraping script to avoid scraping script errors. A predetermined number of webpages targeted by the scraping script are randomly sampled. The scraping script is appended to each webpage in the sample. A structured list of text fragments across the webpages with the appended script is generated. At predetermined time intervals, a fresh set of webpages is sampled, the scraping script is appended to the webpages, and a new structured list is generated. If the new structured list and the previous structured list do not match, the webpages may have been changed and the scraping script may have to be updated. An alert is generated indicating that such update is required and may include a location of the mismatch. Therefore, scraping script errors are proactively detected and can be rectified before an actual error occurs and propagates.

Growing community of inventors

Tel Aviv, Israel

Aleksandr Kim