Hierarchical Model Predictive Control-Guided Reinforcement Learning for Energy Management of Hybrid-Electric Distributed Propulsion Aircraft

This paper proposes a deployable hierarchical control architecture for mission-level energy management in a series hybrid-electric distributed propulsion (DEP) aircraft, explicitly balancing safety and long-horizon optimality. The architecture integrates model predictive control (MPC) with a reinforcement learning-based Soft Actor-Critic (SAC) agent in a phase-adaptive hierarchical structure. MPC ensures conservative, constraint-aware authority during safety-critical phases (takeoff and climb), while the SAC agent provides fuel- and energy-optimal power allocation in less constrained regimes, particularly cruise. Outputs are continuously blended using flight-phase-dependent weights rather than hard switching, with automatic fallback to MPC-only operation upon SAC degradation. The SAC policy, trained off-policy over full missions, generalizes across conditions while strictly enforcing battery state-of-charge and thermal constraints. The energy management is decoupled from trajectory regulation, avoiding online constrained or optimal inner-loop propulsion or speed control. Aircraft velocity evolves naturally from thrust–drag dynamics, guided by a simple fixed-structure feedback controller enforcing predefined phase-dependent speed targets, preserving physical consistency without online constrained optimization. Results demonstrate that the hierarchical MPC-guided SAC framework achieves superior performance across stability, tracking, and efficiency, yielding an overall score of 0.88. By maintaining small SAC corrections between 0.3 and 0.4 in magnitude, the system ensures robust transitions between control authorities. For a 0.38-hour reference mission, the controller leverages low-demand phases for in-flight battery recharging, delivering 9.79 kWh at a peak power of 103 kW to maximize SOC recovery. Compared to MPC-only benchmarks, the strategy achieves a 5.75% reduction in fuel consumption and a 5.7% decrease in
emissions for both Jet-A and sustainable aviation fuel (SAF). These findings establish that the integrated framework provides a robust, real-time solution for reducing emissions and extending battery longevity without compromising flight performance.