论文标题
使用深厚的增强学习，多核电供应链有不确定的季节性需求和交货时间
Multi-echelon Supply Chains with Uncertain Seasonal Demands and Lead Times Using Deep Reinforcement Learning
论文作者
论文摘要
我们解决了多芯供应链中生产计划和分配的问题。我们考虑不确定的需求和交货时间，这使问题随机和非线性。提出了马尔可夫决策过程公式和非线性编程模型。作为一个顺序决策问题，深度加强学习（RL）是一种可能的解决方案方法。近年来，这种技术吸引了人工智能和优化社区的广泛关注。考虑到在不同领域的深入RL方法获得的良好结果，人们对将它们应用于运营研究领域的问题的兴趣越来越大。考虑到不确定，规则和季节性需求以及恒定或随机的交货时间，我们使用了深层RL技术，即近端政策优化（PPO2）。在不同的情况下进行实验，以更好地评估算法的适用性。基于线性化模型的代理被用作基线。实验结果表明，PPO2是解决此类问题的竞争和足够工具。在所有情况下（7.3-11.2％）的所有情况下，PPO2代理都比基线要好，而不管需求是否为季节性。在持续交货时间的情况下，当不确定的需求是非季节（2.2-4.7％）时，PPO2代理会更好。结果表明，场景的不确定性越大，这种方法的可行性越大。
We address the problem of production planning and distribution in multi-echelon supply chains. We consider uncertain demands and lead times which makes the problem stochastic and non-linear. A Markov Decision Process formulation and a Non-linear Programming model are presented. As a sequential decision-making problem, Deep Reinforcement Learning (RL) is a possible solution approach. This type of technique has gained a lot of attention from Artificial Intelligence and Optimization communities in recent years. Considering the good results obtained with Deep RL approaches in different areas there is a growing interest in applying them in problems from the Operations Research field. We have used a Deep RL technique, namely Proximal Policy Optimization (PPO2), to solve the problem considering uncertain, regular and seasonal demands and constant or stochastic lead times. Experiments are carried out in different scenarios to better assess the suitability of the algorithm. An agent based on a linearized model is used as a baseline. Experimental results indicate that PPO2 is a competitive and adequate tool for this type of problem. PPO2 agent is better than baseline in all scenarios with stochastic lead times (7.3-11.2%), regardless of whether demands are seasonal or not. In scenarios with constant lead times, the PPO2 agent is better when uncertain demands are non-seasonal (2.2-4.7%). The results show that the greater the uncertainty of the scenario, the greater the viability of this type of approach.
