(untitled)
This paper quantitatively characterizes the approximation power of deep feed-forward neural networks (FNNs) in terms of the number of neurons. It is shown by construction that ReLU FNNs with width O(maxd⌊N^1/d⌋,N+1) and depth O(L) can approximate an arbitrary Hölder continuous function of order α∈(0,1] on [0,1]^d with a nearly tight approximation rate O(dN^−2α/dL^−2α/d) measured in L^p -norm for any N,L∈N^+ and p∈[1,∞]. More generally for an arbitrary continuous function f on [0,1]^d with a modulus of continuity ω_f (·), the constructive approximation rate is O(dω_f(N^−2α/dL^−2α/d)). We also extend our analysis to f on irregular domains or those localized in an ε-neighborhood of a d_M-dimensional smooth manifold M⊆[0,1]^d with d_M≪d. Especially, in the case of an essentially low-dimensional domain, we show an approximation rate O(ω_f(ε1−δdd_δ +ε)+dω_f(d 1−δd_δ N^−2α/d_δL^−2α/d_δ) for ReLU FNNs to approximate f in the ε-neighborhood, where d_δ=O(d_Mln(d/δ)δ^2) for any δ∈(0,1) as a relative error for a projection to approximate an isometry when projecting M to a d_δ-dimensional domain. , number=5, journal=Communications in Computational Physics, year=2020, month=Nov., pages=1768–1811