Fuzzy Critic Learning for Enhanced Fine-Tuning Policy Optimization

As a policy optimization approach, adaptive critic learning demonstrates theoretical advantages and practical potential. However, existing methods encounter a fundamental challenge in balancing offline and online paradigms, particularly in addressing model uncertainty, parameter variation, and demand adjustment. Specifically, offline learning ensures stable training but lacks adaptability to environmental changes, whereas online learning enables real-time policy adaptation at the expense of data inefficiency and learning oscillation. To address these limitations, we develop an enhanced online fine tuning framework via fuzzy critic learning, which integrates fuzzy rules and data collection with offline knowledge and online adaptability. The proposed method is validated under parameter variations and demand adjustments. Compared with baseline methods, experimental results show that cost-to-go variance is reduced by several orders of magnitude.