MRT4Depth: Metamorphic Robustness Testing for Ground-Truth-Free Evaluation of Monocular Depth Estimation Models

Monocular depth estimation (MDE) must maintain stable predictions under minor input perturbations to ensure robustness and reliability, particularly in safety-critical applications. Existing robustness-evaluation methods rely on comparisons with ground-truth depth data, which are costly to obtain and often unavailable. This limitation constitutes an instance of the oracle problem, where expected outputs for test inputs cannot be determined. To address this challenge, this study introduces an MRT-based framework for evaluating the robustness of MDE models without requiring ground-truth depth annotations (MRT4Depth). Prediction consistency is measured using both pixelwise consistency and pairwise ordinal depth consistency, the latter being specifically designed to capture the relative ordering property inherent in depth estimation. The proposed framework is applied to evaluate four state-of-the-art MDE models, including UniDepthV2, Depth Pro, ZoeDepth, and Depth Anything 3 on 4 benchmark datasets covering indoor and outdoor scenes, using 18 common image corruptions and four Euclidean transformations. The results indicate that the evaluated models exhibit substantially different relative robustness characteristics across perturbation types. The findings further reveal that high prediction accuracy and stable accuracy under perturbations do not necessarily correspond to high prediction consistency. In general, MRT4Depth enables ground-truth-free robustness evaluation and reveals aspects of prediction consistency that accuracy-based robustness metrics do not capture.