Spot Electricity Market Bidding Strategy Optimization Based on Risk Analysis with Reinforcement Learning

Great effort has been made to restructure the traditional monopolization of power industries to introduce fair competition. The deregulation of the electricity market allows the price of electricity to be formulated based on the bidding price. Nevertheless, it is still challenging to derive an optimal bidding strategy with many factors that need to be considered. This paper proposes a reinforcement learning (RL) based method to devise an optimal bidding strategy for maximizing the profit, taking the risk preferences in the spot electricity market into consideration. The problem is formulated based on Markov decision process (MDP), which is a discrete stochastic optimization method. The objective function is to optimize the cumulative profit over the span. This method also employs temporal difference technique and actor-critic learning algorithm for strategy optimization. In addition, the study introduces smart-market market-clearing method and a Gaussian distribution to formulate the strategy. Two different environmental conditions of the spot electricity market, static and dynamic, are applied in the simulation for analysis completeness. Only the target plant can adjust the bidding strategy in the static environment while all plants can adjust the bidding strategy in the dynamic environment. Simulation cases of nine participants are considered and the obtained results are analyzed.