论文标题
绿色模拟辅助加强学习，并具有模型的生物制造风险学习和控制
Green Simulation Assisted Reinforcement Learning with Model Risk for Biomanufacturing Learning and Control
论文作者
论文摘要
生物制药制造面临着关键的挑战，包括复杂性，高可变性，冗长的交货时间以及有限的历史数据以及基础系统随机过程的知识。为了应对这些挑战，我们提出了一个绿色模拟辅助基于模型的增强学习，以支持过程在线学习和指导动态决策。基本上，过程模型风险通过后验分布来量化。在任何给定的政策中，我们都会预测预期的系统响应，并对固有的随机不确定性和模型风险进行预测风险占。然后，我们提出了绿色模拟辅助增强学习，并得出了基于策略梯度的基于决策过程和基于可能性比率的元模型的混合建议分布，该阶段可以选择性地重复使用从以前的实验中收集的过程轨迹输出，以提高模拟数据效率，提高策略梯度估计的准确性，并加快搜索最佳策略的搜索。我们的数值研究表明，所提出的方法证明了有希望的表现。
Biopharmaceutical manufacturing faces critical challenges, including complexity, high variability, lengthy lead time, and limited historical data and knowledge of the underlying system stochastic process. To address these challenges, we propose a green simulation assisted model-based reinforcement learning to support process online learning and guide dynamic decision making. Basically, the process model risk is quantified by the posterior distribution. At any given policy, we predict the expected system response with prediction risk accounting for both inherent stochastic uncertainty and model risk. Then, we propose green simulation assisted reinforcement learning and derive the mixture proposal distribution of decision process and likelihood ratio based metamodel for the policy gradient, which can selectively reuse process trajectory outputs collected from previous experiments to increase the simulation data-efficiency, improve the policy gradient estimation accuracy, and speed up the search for the optimal policy. Our numerical study indicates that the proposed approach demonstrates the promising performance.
