Two-stage deep learning framework for detecting the condition of rail car coupler systems

Abstract

Systems, devices, media, and methods are presented for training a predictive model to detect objects in digital images, such as detecting whether a cotter key is present or absent in a digital photograph of a rail car coupler. The training system includes curating a plurality of training datasets, each including a number of raw images, together with a number of adjusted, augmented, and duplicate images. The predictive model includes a localization algorithm and an ensemble of models for classification. The localization algorithm is a deep convolutional neural network (CNN) which identifies a region of interest. One of more of the deep CNN classification models generates a plurality of candidate regions associated with each region of interest, thereby generating a large number of additional regions useful for training. In use, the trained predictive model is part of a detection and notification system that processes new images from the field and broadcasts a notice when an anomaly is detected.