论文标题
灵活的建模和多任务学习使用可区分的树合奏
Flexible Modeling and Multitask Learning using Differentiable Tree Ensembles
论文作者
论文摘要
决策树的合奏被广泛使用和竞争性学习模型。尽管他们成功了，但流行的学习树合奏的工具包具有有限的建模功能。例如，这些工具包支持有限的损失功能，并且仅限于单个任务学习。我们为学习树的合奏提出了一个灵活的框架，该框架超出了现有工具包，以支持任意损失功能，丢失的响应和多任务学习。我们的框架建立在可区分的（又称软）树的基础上，可以使用一阶方法对其进行训练。但是，与古典树不同，可区分的树木难以扩展。因此，我们提出了一种基于张量的新型树木的新型公式，可在GPU上有效地进行矢量化。我们对28个真正的开源和专有数据集进行了实验，这表明我们的框架可能会导致100倍的紧凑型和23％的富有表现力的树与通过流行工具包的表现力更高。
Decision tree ensembles are widely used and competitive learning models. Despite their success, popular toolkits for learning tree ensembles have limited modeling capabilities. For instance, these toolkits support a limited number of loss functions and are restricted to single task learning. We propose a flexible framework for learning tree ensembles, which goes beyond existing toolkits to support arbitrary loss functions, missing responses, and multi-task learning. Our framework builds on differentiable (a.k.a. soft) tree ensembles, which can be trained using first-order methods. However, unlike classical trees, differentiable trees are difficult to scale. We therefore propose a novel tensor-based formulation of differentiable trees that allows for efficient vectorization on GPUs. We perform experiments on a collection of 28 real open-source and proprietary datasets, which demonstrate that our framework can lead to 100x more compact and 23% more expressive tree ensembles than those by popular toolkits.
