1. 7of9

    Posted

    7of9
Changes in title
+WIP > Paper > Most 2005 (dynardo) > Approximation of complex nonlinear functions by means of neural networks
Changes in tags
Changes in body
Source | HTML | Preview
@@ -0,0 +1,48 @@
+**WIP (Work in Progress)**
+
+[Approximation of complex nonlinear functions by means of neural networks](https://www.dynardo.de/fileadmin/Material_Dynardo/WOST/Paper/wost2.0/NeuralNetworks.pdf)
+Thomas Most
+presented at the 2nd Weimar Optimization and Stochastic Days 2005
+
+2017年現在からすると、12年も前の論文。
+12年の間にこの論文内の内容について新しい発見があるかもしれない点は注意事項。
+
+いくつかの問題に対して、mutli-layer perceptron (also called feed-forward back-propagation network)を用いてfunction approximationを行っている。
+
+以下は気になったキーワードの抜粋。
+
+- 1. Introduction
+ - Moving Least Squared: Lancaster and Salkauskas (1981)
+ - ANN: artificial neural networks
+ - A good overview: Hagan et al. (1996)
+ - RSM
+ - limited to problems of lower dimension
+ - Lehky´ and Nov´ak (2004)
+ - material parameters of a smeared crack model for concrete cracking were identified
+- 2. Neural network approximation
+ - Eq.1: output of a single neuron
+ - a_i^j
+ - m, i, j, w_k,i^j
+ - sigmoid transfer function
+ - linear output layer with f(x) = x
+ - Demuth and Beale (2002): a complete list of different transfer functions
+ - Three points: influence on the approximation quality of NN
+ - training
+ - design of the network architecture
+ - choice of appropriate training sample
+ - Scaled Conjugate Gradient: Møller (1993)
+ - Eq.2: number of training samples m should be ... [link](http://qiita.com/7of9/items/c1501c2c3fb58df0f445)
+ - regularization
+ - Bayesian training: MacKay (1992)
+ - very slow for higher dimensions compared to Newton-based training approaches.
+ - does not avoid over-fitting completely
+ - early stopping
+ - needs additional data set, the control set
+ - stops if the control set starts to increase, while the training error decreases
+ - does not avoid over-fitting completely
+ - Papadrakakis et al. (1996), Hurtado (2002)
+ - training samples are generated by standard Monte Calro Simulation
+ - Lehky´ and Nov´ak (2004)
+ - **LHS: Latin Hypercube Sampling**
+ - [link @ lucille 開発日記](http://lucille.sourceforge.net/blog/archives/000188.html)
+