论文标题
通过暹罗代表学习360度视频中的全向流量
Learning Omnidirectional Flow in 360-degree Video via Siamese Representation
论文作者
论文摘要
全向视频中的光流估计面临两个重要问题：缺乏基准数据集以及调整基于视频视频的方法以适应全向性质的挑战。本文提出了第一个具有360度视野Flow360的感知上天然合成的全向基准数据集，其中有40个不同的视频和4,000个视频帧。我们在数据集和现有的光流数据集之间进行了全面的特征分析和比较，这些数据集表现出感知现实主义，独特性和多样性。为了适应全向性质，我们提出了一个新颖的暹罗表示学习框架（SLOF）。我们以对比度的方式训练我们的网络，并结合了对比度损耗和光流损失的混合损失函数。广泛的实验验证了所提出的框架的有效性，并在最先进的方法中显示出40％的性能提高。我们的Flow360数据集和代码可在https://siamlof.github.io/上找到。
Optical flow estimation in omnidirectional videos faces two significant issues: the lack of benchmark datasets and the challenge of adapting perspective video-based methods to accommodate the omnidirectional nature. This paper proposes the first perceptually natural-synthetic omnidirectional benchmark dataset with a 360-degree field of view, FLOW360, with 40 different videos and 4,000 video frames. We conduct comprehensive characteristic analysis and comparisons between our dataset and existing optical flow datasets, which manifest perceptual realism, uniqueness, and diversity. To accommodate the omnidirectional nature, we present a novel Siamese representation Learning framework for Omnidirectional Flow (SLOF). We train our network in a contrastive manner with a hybrid loss function that combines contrastive loss and optical flow loss. Extensive experiments verify the proposed framework's effectiveness and show up to 40% performance improvement over the state-of-the-art approaches. Our FLOW360 dataset and code are available at https://siamlof.github.io/.
