论文标题
用于稀疏多光谱差异的暹罗CNNS
Domain Siamese CNNs for Sparse Multispectral Disparity Estimation
论文作者
论文摘要
由于许多原因，多光谱差异估计是一项艰巨的任务：除了图像之间的图像之间几乎没有常见的视觉信息（例如，颜色信息与热信息信息与热信息信息），它与传统可见可见的差异估计（遮挡，重复模式，无纹理表面）所面临的所有挑战。在本文中，我们提出了一种新的CNN体系结构，能够在不同频谱的图像之间进行差异估计，即在我们的情况下可见。我们提出的模型将两个补丁作为输入进行，并继续为每个贴片进行域特征提取。然后将两个域的特征与两个融合操作合并，即相关性和串联。然后将这些合并的向量转发到其各自的分类头，这些媒介负责将输入分类为相同与否。使用两个合并操作可以使我们的特征提取过程更加鲁棒，这会导致更精确的差异估计。我们的方法使用了公开可用的LITIV 2014和LITIV 2018数据集进行了测试，与其他最先进的方法相比，它显示出最佳的结果。
Multispectral disparity estimation is a difficult task for many reasons: it has all the same challenges as traditional visible-visible disparity estimation (occlusions, repetitive patterns, textureless surfaces), in addition of having very few common visual information between images (e.g. color information vs. thermal information). In this paper, we propose a new CNN architecture able to do disparity estimation between images from different spectrum, namely thermal and visible in our case. Our proposed model takes two patches as input and proceeds to do domain feature extraction for each of them. Features from both domains are then merged with two fusion operations, namely correlation and concatenation. These merged vectors are then forwarded to their respective classification heads, which are responsible for classifying the inputs as being same or not. Using two merging operations gives more robustness to our feature extraction process, which leads to more precise disparity estimation. Our method was tested using the publicly available LITIV 2014 and LITIV 2018 datasets, and showed best results when compared to other state of the art methods.
