Long short-term memory network with adapted attention mechanism for credit risk modeling

In recent years, survival models have received increasing attention in credit risk. Unlike classification models typically used to model defaults, survival analysis can model not only whether a borrower will default but also the time to default. Since loan transaction data are discrete time-series information, it is natural to apply a discrete-time survival model (DTSM). At the same time, there have been significant advances in deep neural networks. In this paper, we extend the DTSM using long short-term memory (LSTM) networks, incorporating an adapted LSTM-based attention mechanism to better uncover temporal features that influence the probability of default, along with a washout phase, which is used to iterate several dummy LSTM prediction steps and hence addresses the LSTM state initialization problem. Finally, instead of using a standard attention mechanism that linearly encodes the input, we adapt it with an LSTM layer that captures non-linear temporal features from the sequential data. This model shows great improvement in model fit and predictive ability, in comparison with baseline linear DTSMs and standard LSTM with attention, when evaluated on US mortgage data. Moreover, we show that our proposed model gives good forecast performance, providing practitioners with a practical and powerful early warning service to manage potential credit loss.