论文标题
二次神经网络均匀近似
Uniform Approximation with Quadratic Neural Networks
论文作者
论文摘要
在这项工作中，我们检查了使用整流的二次单元（requ）激活函数（定义为\（\ max（0，x）^2 \）的深神网络的近似能力，以近似于Hölder-norkular函数相对于均匀的规范。我们建设性地证明，具有重新激活的深神经网络可以近似\（r \） - \（r \） - （r \） - hölder-regular-regular functions（\（\ Mathcal {h}^{h}^{r，r，r，r，r，r，r，r，r}（[ - 1,1]^d）\）的任何功能，最多是任何精确度\（最多）。 \（\ Mathcal {O} \ left（ε^{ - d /2r} \ right）\）\）\）\固定的层。该结果强调，近似的有效性显着取决于目标函数的平滑性和必需函数的特征。我们的证明是基于近似泰勒的近似泰勒扩展具有深厚的必要神经网络的基础，证明了它们有效捕获Hölder规范功能的行为的能力。此外，对于\（\ max（0，x）^p \）的任何整流动力单元（repu）激活函数的结果可以直接概括为\（p \ geq 2 \），这表明我们发现在这个激活家族中我们发现的更广泛适用性。
In this work, we examine the approximation capabilities of deep neural networks utilizing the Rectified Quadratic Unit (ReQU) activation function, defined as \(\max(0,x)^2\), for approximating Hölder-regular functions with respect to the uniform norm. We constructively prove that deep neural networks with ReQU activation can approximate any function within the \(R\)-ball of \(r\)-Hölder-regular functions (\(\mathcal{H}^{r, R}([-1,1]^d)\)) up to any accuracy \(ε\) with at most \(\mathcal{O}\left(ε^{-d /2r}\right)\) neurons and fixed number of layers. This result highlights that the effectiveness of the approximation depends significantly on the smoothness of the target function and the characteristics of the ReQU activation function. Our proof is based on approximating local Taylor expansions with deep ReQU neural networks, demonstrating their ability to capture the behavior of Hölder-regular functions effectively. Furthermore, the results can be straightforwardly generalized to any Rectified Power Unit (RePU) activation function of the form \(\max(0,x)^p\) for \(p \geq 2\), indicating the broader applicability of our findings within this family of activations.
