TensorFlow / difference > tf.nn.sparse_softmax_cross_entropy_with_logits() / tf.nn.softmax_cross_entropy_with_logits() > exclusive / soft claesses

動作環境

Ubuntu 14.04 LTS desktop amd64
GeForce GTX 750 Ti
ASRock Z170M Pro4S [Intel Z170chipset]
TensorFlow v0.11
cuDNN v5.1 for Linux
CUDA v7.5
Python 2.7.6
IPython 5.1.0 -- An enhanced Interactive Python.

mnist.pyの以下が気になった。

mnist.py

...
  cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
      logits, labels, name='xentropy')

...

cross entropyの計算だが、sparse_が付いている。

NOTE: For this operation, the probability of a given label is considered exclusive. That is, soft classes are not allowed, and the labels vector must provide a single specific index for the true class for each row of logits (each minibatch entry). For soft softmax classification with a probability distribution for each entry, see softmax_cross_entropy_with_logits.

NOTE: While the classes are mutually exclusive, their probabilities need not be. All that is required is that each row of labels is a valid probability distribution. If they are not, the computation of the gradient will be incorrect.

If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits.

exclusive, soft claessesの意味はそのうち分かるだろう。