论文标题
通过基于光流的QoE的增强学习驱动自适应VR流媒体流
Reinforcement Learning Driven Adaptive VR Streaming with Optical Flow Based QoE
论文作者
论文摘要
在一个相机中包含完整的全景内容的优点，虚拟现实（VR）和360度视频在工业云制造和培训领域吸引了越来越多的关注。工业互联网（IoT），许多VR终端都需要同时在线，几乎不能保证VR的带宽要求。但是，通过利用用户的经验质量（QOE）意识因素，包括观点和其他内容之间的相对移动速度和深度差异，可以减少带宽消耗。在本文中，我们提出了B-VR（基于光流的VR），这是一种VR流的交互方法，可以利用VR用户的QOE意识来减轻带宽压力。探索了通过光流估计（JND-OFE）的差异差异，以量化用户在360度视频中对质量失真的认识。因此，提出了基于PSNR和JND-OFE（PSNR）的新型360度视频QOE度量。在PSNR的帮助下，OFB-VR提出了一种多功能大小的平铺方案，以减少铺平的开销。实施了增强学习（RL）方法来利用历史数据来执行自适应比特率（ABR）。为了进行评估，我们将两个先前的VR流媒体方案（Pano和Plato）作为基础。广泛的评估表明，我们的系统可以将平均PSNR得分提高9.5-15.8％，同时与Pano和Plato在波动的LTE带宽数据集中保持相同的拒绝比率。评估结果表明，OFB-VR是实际交互式工业VR的有希望的原型。可以在https://github.com/buptexplorers/ofb-vr中找到OFB-VR的原型。
With the merit of containing full panoramic content in one camera, Virtual Reality (VR) and 360-degree videos have attracted more and more attention in the field of industrial cloud manufacturing and training. Industrial Internet of Things (IoT), where many VR terminals needed to be online at the same time, can hardly guarantee VR's bandwidth requirement. However, by making use of users' quality of experience (QoE) awareness factors, including the relative moving speed and depth difference between the viewpoint and other content, bandwidth consumption can be reduced. In this paper, we propose OFB-VR (Optical Flow Based VR), an interactive method of VR streaming that can make use of VR users' QoE awareness to ease the bandwidth pressure. The Just-Noticeable Difference through Optical Flow Estimation (JND-OFE) is explored to quantify users' awareness of quality distortion in 360-degree videos. Accordingly, a novel 360-degree videos QoE metric based on PSNR and JND-OFE (PSNR-OF) is proposed. With the help of PSNR-OF, OFB-VR proposes a versatile-size tiling scheme to lessen the tiling overhead. A Reinforcement Learning(RL) method is implemented to make use of historical data to perform Adaptive BitRate(ABR). For evaluation, we take two prior VR streaming schemes, Pano and Plato, as baselines. Vast evaluations show that our system can increase the mean PSNR-OF score by 9.5-15.8% while maintaining the same rebuffer ratio compared with Pano and Plato in a fluctuate LTE bandwidth dataset. Evaluation results show that OFB-VR is a promising prototype for actual interactive industrial VR. A prototype of OFB-VR can be found in https://github.com/buptexplorers/OFB-VR.
