Suche nach Personen

plus im Publikationsserver
plus bei BASE
plus bei Google Scholar

Daten exportieren

 

Approximation Spaces of Deep Neural Networks

Titelangaben

Verfügbarkeit überprüfen

Gribonval, Rémi ; Kutyniok, Gitta ; Nielsen, Morten ; Voigtlaender, Felix:
Approximation Spaces of Deep Neural Networks.
In: Constructive approximation. 55 (2022). - S. 259-367.
ISSN 0176-4276 ; 1432-0940

Volltext

Open Access
Volltext Link zum Volltext (externe URL):
https://doi.org/10.1007/s00365-021-09543-4

Kurzfassung/Abstract

We study the expressivity of deep neural networks. Measuring a network’s complexity by its number of connections or by its number of neurons, we consider the class of functions for which the error of best approximation with networks of a given complexity decays at a certain rate when increasing the complexity budget. Using results from classical approximation theory, we show that this class can be endowed with a (quasi)-norm that makes it a linear function space, called approximation space. We establish that allowing the networks to have certain types of “skip connections” does not change the resulting approximation spaces. We also discuss the role of the network’s nonlinearity (also known as activation function) on the resulting spaces, as well as the role of depth. For the popular ReLU nonlinearity and its powers, we relate the newly constructed spaces to classical Besov spaces. The established embeddings highlight that some functions of very low Besov smoothness can nevertheless be well approximated by neural networks, if these networks are sufficiently deep.

Weitere Angaben

Publikationsform:Artikel
Sprache des Eintrags:Englisch
Institutionen der Universität:Mathematisch-Geographische Fakultät > Mathematik > Lehrstuhl für Mathematik - Wissenschaftliches Rechnen/Informatik
Mathematisch-Geographische Fakultät > Mathematik > Lehrstuhl für Reliable Machine Learning
DOI / URN / ID:10.1007/s00365-021-09543-4
Open Access: Freie Zugänglichkeit des Volltexts?:Ja
Peer-Review-Journal:Ja
Verlag:Springer
Die Zeitschrift ist nachgewiesen in:
Titel an der KU entstanden:Ja
KU.edoc-ID:26832
Eingestellt am: 14. Jun 2021 14:28
Letzte Änderung: 12. Mai 2022 08:59
URL zu dieser Anzeige: https://edoc.ku.de/id/eprint/26832/
AnalyticsGoogle Scholar