【Applied Energy最新原创论文】一种用于调查和优化端对端产消者能源市场的多智能体强化学习方法

AEii国际应用能源 · 公众号 · · 2023-02-11 18:30

正文

原文信息：

A multi-agent reinforcement learning approach for investigating and optimising peer-to-peer prosumer energy markets

原文链接：

https://www.sciencedirect.com/science/article/pii/S0306261923000697

Highlights

• Multi-agent reinforcement learning has been used for optimising the energy sharing market.

• The developed strategy respects data privacy and requires no data sharing between prosumers.

• Emergent social–economic behaviour such as price elasticity has been observed.

• The learned dynamic price policy outperforms a benchmark fixed price strategy.

• Compared to the fixed price strategy, community net profit increased by 28.64%.

摘要

当前的电网基建在设计阶段并没有气候变化考虑在内，因此，其稳定性受到了影响（尤其在需求高峰期）。此外，鉴于当前联合国政府间气候变化专门委员会关于全球变暖的报告以及2015年巴黎气候协议的目标，即需要采取紧急的社会技术措施，将全球平均气温较前工业化时期上升幅度控制在1.5-2℃的范围内。智能微电网和可再生能源技术已被认为是帮助缓解全球变暖和电网不稳定的可能解决方案。在此背景下，合理管理的需求侧灵活性对于高效的就地利用太阳能至关重要。为此，一个设计合理的动态定价机制可以组织此类系统内的参与者，以实现现场能源的有效交易，从而有助于实现上述脱碳和电网安全目标。然而，在如此复杂和动态的经济环境中设计这样一个机制，往往会导致计算上难以解决的解决方案。为了克服这个问题，本文将多智能体强化学习与Foundation（一种由Salesforce Research构建的开源经济模拟框架）结合的方法来设计动态价格政策。通过将具有异质需求/供能配置和电池储能的端对端产消者社区纳入Foundation。我们从数据驱动模拟的结果表明，与基准固定价格信号相比，多智能体强化学习可以学习动态价格信号，从而实现降低社区电力成本和提高社区自给自足率。此外，还确定了一些突发的社会经济行为，例如价格弹性和社区协调，导致在光伏供应过剩期间从市政电网购电。本文提出的方法可以被从业者用来帮助他们设计端对端能源交易市场。

更多关于“Rreinforcement learning”的研究请见：

https://www.sciencedirect.com/search?pub=Applied%20Energy&cid=271429&qs=reinforcement%20learning

Abstr act

Current power grid infrastructure was not designed with climate change in mind, and, therefore, its stability, especially at peak demand periods, has been compromised. Furthermore, in light of the current UN’s Intergovernmental Panel on Climate Change reports concerning global warming and the goal of the 2015 Paris climate agreement to constrain global temperature increase to within 1.5–2 ℃ above pre-industrial levels, urgent sociotechnical measures need to be taken. Together, Smart Microgrid and renewable energy technology have been proposed as a possible solution to help mitigate global warming and grid instability. Within this context, well-managed demand-side flexibility is crucial for efficiently utilising on-site solar energy. To this end, a well-designed dynamic pricing mechanism can organise the actors within such a system to enable the efficient trade of on-site energy, therefore contributing to the decarbonisation and grid security goals alluded to above. However, designing such a mechanism in an economic setting as complex and dynamic as the one above often leads to computationally intractable solutions. To overcome this problem, in this work, we use multi-agent reinforcement learning (MARL) alongside Foundation – an open-source economic simulation framework built by Salesforce Research – to design a dynamic price policy. By incorporating a peer-to-peer (P2P) community of prosumers with heterogeneous demand/supply profiles and battery storage into Foundation, our results from data-driven simulations show that MARL, when compared with a baseline fixed price signal, can learn a dynamic price signal that achieves both a lower community electricity cost, and a higher community self-sufficiency. Furthermore, emergent social–economic behaviours, such as price elasticity, and community coordination leading to high grid feed-in during periods of overall excess photovoltaic (PV) supply and, conversely, high community trading during overall low PV supply, have also been identified. Our proposed approach can be used by practitioners to aid them in designing P2P energy trading markets.

Keywords

Peer-to-peer market

Community-based market

Dynamic pricing

Multi-agent systems

Multi-agent reinforcement learning

Proximal Policy Optimisation

【Applied Energy最新原创论文】一种用于调查和优化端对端产消者能源市场的多智能体强化学习方法

正文

请到「今天看啥」查看全文