System and method for code smell detection using transformer-based code representations with self-supervision by predicting reserved words

Abstract

A device, method, and non-transitory computer readable medium that for analyzing computer source code to detect code smells is disclosed. The method includes inputting, via processing circuitry, the source code and creating, via the processing circuitry, pseudo labels by a proxy task based on a vector of tokens for the source code. In addition, the method includes training, via the processing circuitry, a transformer model on the pseudo labels, as a pre-trained model that outputs a prediction of a value of tokens in the vector of tokens, and applying, via the processing circuitry, the pre-trained model to a plurality of fine-tuning models for respective downstream tasks, where each fine-tuning model is created by training the pre-trained model. The method also includes outputting, via the processing circuitry, from each fine-tuning model, an indication of whether a code smell has been detected in the source code.