Stanfordの授業CS231nでニューラルネット、畳み込みニューラルネットを学ぶ

StanfordのCS231nという授業の教材を使って、機械学習を学んだ。
自分のメモのまとめ。
（写経に近いので注意）

ちなみにこのCS231nは「ゼロから作るDeepLearning」という本の元ネタのよう。

Module 1: Neural Networks

• Image Classification: Data-driven Approach, k-Nearest Neighbor, train/val/test splits
  • L1/L2 distances, hyperparameter search, cross-validation
• Linear classification: Support Vector Machine, Softmax
  • parameteric approach, bias trick, hinge loss, cross-entropy loss, L2 regularization, web demo
• Optimization: Stochastic Gradient Descent
  • optimization landscapes, local search, learning rate, analytic/numerical gradient
• Backpropagation, Intuitions
  • chain rule interpretation, real-valued circuits, patterns in gradient flow
• Neural Networks Part 1: Setting up the Architecture
  • model of a biological neuron, activation functions, neural net architecture, representational power
• Neural Networks Part 2: Setting up the Data and the Loss
  • preprocessing, weight initialization, batch normalization, regularization (L2/dropout), loss functions
• Neural Networks Part 3: Learning and Evaluation
  • gradient checks, sanity checks, babysitting the learning process, momentum (+nesterov), second-order methods, Adagrad/RMSprop, hyperparameter optimization, model ensembles
• Putting it together: Minimal Neural Network Case Study
  • minimal 2D toy data example

Module 2: Convolutional Neural Networks（更新予定）

• Convolutional Neural Networks: Architectures, Convolution / Pooling Layers
  • layers, spatial arrangement, layer patterns, layer sizing patterns, AlexNet/ZFNet/VGGNet case studies, computational considerations
• Understanding and Visualizing Convolutional Neural Networks
  • tSNE embeddings, deconvnets, data gradients, fooling ConvNets, human comparisons
• Transfer Learning and Fine-tuning Convolutional Neural Networks