Suche nach Personen

plus im Publikationsserver
plus bei BASE
plus bei Google Scholar

Daten exportieren

 

Topological properties of the set of functions generated by neural networks of fixed size

Titelangaben

Verfügbarkeit überprüfen

Petersen, Philipp ; Raslan, Mones ; Voigtlaender, Felix:
Topological properties of the set of functions generated by neural networks of fixed size.
2018. - 51 S.

Volltext

Open Access
Volltext Link zum Volltext (externe URL):
https://arxiv.org/abs/1806.08459

Kurzfassung/Abstract

We analyze the topological properties of the set of functions that can be implemented by neural networks of a fixed size. Surprisingly, this set has many undesirable properties. It is highly non-convex, except possibly for a few exotic activation functions. Moreover, the set is not closed with respect to Lp-norms, 0<p<∞, for all practically-used activation functions, and also not closed with respect to the L∞-norm for all practically-used activation functions except for the ReLU and the parametric ReLU. Finally, the function that maps a family of weights to the function computed by the associated network is not inverse stable for every practically used activation function. In other words, if f1,f2 are two functions realized by neural networks and if f1,f2 are close in the sense that ∥f1−f2∥L∞≤ε for ε>0, it is, regardless of the size of ε, usually not possible to find weights w1,w2 close together such that each fi is realized by a neural network with weights wi. Overall, our findings identify potential causes for issues in the training procedure of deep learning such as no guaranteed convergence, explosion of parameters, and slow convergence.

Weitere Angaben

Publikationsform:Preprint, Working paper, Diskussionspapier
Sprache des Eintrags:Englisch
Institutionen der Universität:Mathematisch-Geographische Fakultät > Mathematik > Lehrstuhl für Mathematik - Wissenschaftliches Rechnen/Informatik
Mathematisch-Geographische Fakultät > Mathematik > Lehrstuhl für Reliable Machine Learning
DOI / URN / ID:arXiv:1806.08459
Open Access: Freie Zugänglichkeit des Volltexts?:Ja
Titel an der KU entstanden:Ja
KU.edoc-ID:23471
Eingestellt am: 22. Okt 2019 14:29
Letzte Änderung: 31. Mär 2022 16:51
URL zu dieser Anzeige: https://edoc.ku.de/id/eprint/23471/
AnalyticsGoogle Scholar