论文标题
Axiou：视频矩检索的公理合理的措施
AxIoU: An Axiomatically Justified Measure for Video Moment Retrieval
论文作者
论文摘要
评估措施对研究方向有至关重要的影响。因此，对于不适合常规措施的新应用程序制定适当可靠的评估措施至关重要。视频力矩检索（VMR）就是这样的应用程序，当前的做法是使用r@$ k，θ$来评估VMR系统。但是，该措施有两个缺点。首先，它是不敏感的：它通过将列表视为一组，忽略了在顶部$ k $排名列表中成功本地化时刻的排名位置。其次，它使用阈值$θ$将每个检索到的视频时刻的联合（IOU）交叉分配，从而忽略了排名时刻的细粒度本地化质量。 我们提出了一种评估VMR的替代措施，称为平均最大IOU（AXIOU），该措施脱离了以上两个问题。我们表明，Axiou满足了VMR评估的两个重要公理，即\ textbf {针对冗余矩}的不变性}和\ textbf {单调相对于最佳时刻}，而r@$ k，θ$也只能满足第一个辅助功能。我们还经验研究Axiou如何同意R@$ K，θ$，以及其在测试数据和人类宣布的时间边界方面的稳定性。
Evaluation measures have a crucial impact on the direction of research. Therefore, it is of utmost importance to develop appropriate and reliable evaluation measures for new applications where conventional measures are not well suited. Video Moment Retrieval (VMR) is one such application, and the current practice is to use R@$K,θ$ for evaluating VMR systems. However, this measure has two disadvantages. First, it is rank-insensitive: It ignores the rank positions of successfully localised moments in the top-$K$ ranked list by treating the list as a set. Second, it binarizes the Intersection over Union (IoU) of each retrieved video moment using the threshold $θ$ and thereby ignoring fine-grained localisation quality of ranked moments. We propose an alternative measure for evaluating VMR, called Average Max IoU (AxIoU), which is free from the above two problems. We show that AxIoU satisfies two important axioms for VMR evaluation, namely, \textbf{Invariance against Redundant Moments} and \textbf{Monotonicity with respect to the Best Moment}, and also that R@$K,θ$ satisfies the first axiom only. We also empirically examine how AxIoU agrees with R@$K,θ$, as well as its stability with respect to change in the test data and human-annotated temporal boundaries.
