A diagnostic framework for imbalanced classification in business process predictive monitoring

Abstract

One of the use cases of business process predictive monitoring is predicting the next activity in a running case, which results in a multi-class classification problem. Approaches to this use case are usually evaluated considering average performance across all classes. This often masks poor performance on minority classes, particularly when classes to be predicted are imbalanced. This is the natural case in next activity prediction, where exceptions or optional activities occur, by design, less frequently than others. In this paper we propose a framework to diagnose poor predictive performance on the minority class in the next activity prediction use case that comprises two tools: an empirical comparison of different resampling techniques in the data preparation phase and a novel classification performance measure. The proposed performance measure aims at highlighting the poor recall on the minority class of a classifier, which is a particularly important performance in the context of next activity prediction, whereas the benchmark helps understanding which resampling technique would be the best at mitigating the poor recall. We also discuss how the two tools of the proposed framework can be combined from an AutoML perspective. The proposed framework has been evaluated on a set of publicly available event logs.