Uncertainty-aware neural network pruning

Abstract

Neural network pruning is a popular approach to reducing model capac ity and computation by removing redundant parameters from the neural network, however, it faces three critical limitations. First, current research work evaluates the performance of pruned models solely based on the ac curacy of point estimates. The uncertainty of the predictions from the pruned models remains unexplored. In safety-critical applications, uncer tainty plays a vital role in enhancing the trustworthiness of deep learning models and influencing decision-making. Second, existing neural pruning methods remove parameters based on a predefined criterion. These prun ing criteria are heuristic and not related to predictive uncertainty. Third, in continual learning, pruning-based methods often lead to monotonically growing network capacity.

To overcome the first challenge, we employ Inductive Conformal Pre diction (ICP) to provide rigorous uncertainty quantification, demonstrating that pruned models can maintain validity and predictive efficiency. For the second challenge, we introduce a novel pruning criterion that directly opti mizes for uncertainty. To scale this to deep networks, we develop an efficient approximation using first-order Taylor expansion and conformal training, incorporating a protective mechanism to prevent layer collapse. Finally, to enable fixed-capacity continual learning, we propose a method based on horseshoe Bayesian neural networks, which induces sparsity to identify compact task-specific subnetworks, achieving zero catastrophic forgetting.

In summary, this thesis significantly advances the trustworthiness and efficiency of pruned models. Our contributions include: 1) a framework for quantifying predictive uncertainty via ICP; 2) a new class of pruning algo rithms that directly optimize this uncertainty, efficiently scaled to CNNs; and 3) aBayesian continual learning method that prevents forgetting within a fixed capacity. We empirically show that high levels of sparsity (e.g., 95% for MLPs, 80% for CNNs) can be achieved without compromising uncertainty estimates, paving the way for greener and more reliable deep learning.

Date of Award	15 May 2026
Original language	English
Awarding Institution	University of Nottingham
Supervisor	Anthony Graham Bellotti (Supervisor), Jiawei Li (Supervisor) & Amin Farjudian (Supervisor)