Reinforcement Learning Based Optimal Bidding Strategy Learning Framework With Risk Preference in Spot Electricity Market

Great effort has been made to restructure the traditional monopoly power industry, introducing fair competition. The deregulation of market allows the electricity price to form based on power plants offers indicating generation willing at corresponding bidding price. This paper proposes reinforcement
learning(RL) methods to devise optimal bidding strategy maximizing the profit with consideration of risk preference in spot electricity market. The problem is formulated in the framework of Markov decision process (MDP), a discrete stochastic optimization method. The cumulative profit over the span is the objective function to be optimized. The temporal difference technique and actor-critic learning algorithm are employed. The Smart-Market market-clearing system and Gaussian distribution is included in the formulation. Two different environment conditions of the spot electricity market, static and dynamic, are applied in simulation for analysis completeness. Only the target plant can adjust bidding strategy in the static environment while all plants can adjust bidding strategy in the dynamic environment. Simulation cases of nine participants are considered and the obtained results are analyzed.