Neural Network > link (2010) > How to choose the number of hidden layers and nodes in a feedforward neural network? > pruning | { Machine Mode | Regression Mode } | one hidden layer | between the size of the input and size of the output layers

More than 1 year has passed since last update.
GeForce GTX 1070 (8GB)
ASRock Z170M Pro4S [Intel Z170chipset]
Ubuntu 16.04 LTS desktop amd64
TensorFlow v1.1.0
cuDNN v5.1 for Linux
CUDA v8.0
Python 3.5.2
IPython 6.0.0 -- An enhanced Interactive Python.
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)

answered Aug 2 '10 at 2:20


one family of these works by pruning nodes based on (small) values of the weight vector after a certain number of training epochs--in other words, eliminating unnecessary/redundant nodes (more on this below).


Machine Mode| Regression Mode

Is your NN going running in Machine Mode or Regression Mode (the ML convention of using a term that is also used in statistics but assigning a different meaning to it is very confusing). Machine mode: returns a class label (e.g., "Premium Account"/"Basic Account"). Regression Mode returns a value (e.g., price).

one hidden layer

One issue within this subject on which there is a consensus is the performance difference from adding additional hidden layers: the situations in which performance improves with a second (or third, etc.) hidden layer are very few. One hidden layer is sufficient for the large majority of problems.

number of nodes of the hidden layer

So what about size of the hidden layer(s)--how many neurons? There are some empirically-derived rules-of-thumb, of these, the most commonly relied on is 'the optimal size of the hidden layer is usually between the size of the input and size of the output layers'. Jeff Heaton, author of Introduction to Neural Networks in Java offers a few more.

Regression Modeでも(between the size of the input and size of the output layers)で正しいか疑問に思う。

2017年8月16日現在、TensorFlowで精度を調べているものでは30x100x100のhidden layerが一番良い結果を出している(が希望の精度に達してはいない)。
5 input nodes, 6 output nodesに対して(5+6)/2nodesでは希望精度を出せるようには思わない。