MP-DistillFormer: Multimodal prototype distillation with self-supervised transformer for few-shot fine-grained classification

Few-shot fine-grained image classification remains challenging due to limited supervision and the need to distinguish subtle differences among visually similar categories. We propose MP-DistillFormer, a three-stage framework that enhances discrimination and generalization through multimodal prototype distillation and self-supervised refinement. In the first stage, a CNN-based teacher is trained with stochastic augmentation to capture diverse local features. In the second stage, knowledge is distilled into our proposed TeSMo-KAN student model. TeSMo-KAN unifies convolutional tokenization for spatial precision, lightweight local refinement for detail preservation, and nonlinear decision modeling within a Transformer backbone. To enrich semantic representation, TeSMo-KAN is guided by multimodal prototypes that fuse complementary visual and textual embeddings, enabling more discriminative learning under few-shot settings. Finally, a rotation-based self-supervised fine-tuning stage improves robustness under data-scarce conditions. Extensive experiments on three fine-grained benchmarks including CUB-200-2011, Stanford Dogs, and Stanford Cars demonstrate that MP-DistillFormer consistently outperforms state-of-the-art methods in both 1-shot and 5-shot scenarios. The source code is available at https://github.com/annym-ai00/MP-DistillFormer.