System, method, and computer-readable medium that facilitate in-database analytics with supervised data discretization

Abstract

A system, method, and computer-readable medium that facilitate in-database supervised discretisation mechanisms which improve data classification are provided. The disclosed mechanisms provide an efficient, automatic, and repeatable way to perform data discretisation without human intervention. Efficient processing of large and complex unknown data is provided that advantageously does not require the data being analyzed to be processed outside the database. The disclosed mechanisms may use an External Stored Procedure to avoid multiple joins of large tables and minimize the number of full table scans and, consequently, provide better performance than contemporary mechanisms. The disclosed system produces intermediate results in tables which may be conveyed to a visualization subsystem thereby providing users a better understanding of the data distribution in each category. Further, the disclosed system and method introduce a novel similarity-based solution to merge intervals when chi-square testing is not reliable and thereby improves the quality of the interval merge process.