Using a double-blind challenge to evaluate machine-learning-based prognostic-surveillance techniques

Abstract

A double-blind comparison is performed between prognostic-surveillance systems, which are located on a local system and a remote system. During operation, the local system inserts random faults into a dataset to produce a locally seeded dataset, wherein the random faults are inserted into random signals at random times with variable fault signatures. Next, the local system exchanges the locally seeded dataset with a remote system, and in return receives a remotely seeded dataset, which was produced by the remote system by inserting different random faults into the same dataset. Next, the local system uses a local prognostic-surveillance system to analyze the remotely seeded dataset to produce locally detected faults. Finally, the local system determines a performance of the local prognostic-surveillance system by comparing the locally detected faults against actual faults in the remotely seeded fault information. The remote system similarly determines a performance of a remote prognostic-surveillance system.