论文标题
局部优化通常在符号回归的基因编程中不适合
Local Optimization Often is Ill-conditioned in Genetic Programming for Symbolic Regression
论文作者
论文摘要
已经证明基于梯度的局部优化可以改善符号回归的遗传编程（GP）的结果。几种最先进的GP实现使用了迭代非线性最小二乘（NLS）算法，例如Levenberg-Marquardt算法进行局部优化。 NLS算法的有效性取决于优化问题的适当缩放和条件。到目前为止，这在符号回归和GP文献中被忽略了。在这项研究中，我们使用NLS Jacobian矩阵的奇异值分解来确定数字等级和条件数。我们使用GP实施和六个不同的基准数据集执行实验。我们的结果表明，缺乏等级和条件不足的雅各布矩阵经常发生，并且对于所有数据集。当限制GP树大小以及在函数集中使用许多非线性函数时，问题并不那么极端。
Gradient-based local optimization has been shown to improve results of genetic programming (GP) for symbolic regression. Several state-of-the-art GP implementations use iterative nonlinear least squares (NLS) algorithms such as the Levenberg-Marquardt algorithm for local optimization. The effectiveness of NLS algorithms depends on appropriate scaling and conditioning of the optimization problem. This has so far been ignored in symbolic regression and GP literature. In this study we use a singular value decomposition of NLS Jacobian matrices to determine the numeric rank and the condition number. We perform experiments with a GP implementation and six different benchmark datasets. Our results show that rank-deficient and ill-conditioned Jacobian matrices occur frequently and for all datasets. The issue is less extreme when restricting GP tree size and when using many non-linear functions in the function set.
