StanfordのCS231nという授業の教材を使って、機械学習を学んだ。
自分のメモのまとめ。
（写経に近いので注意）
ちなみにこのCS231nは「ゼロから作るDeepLearning」という本の元ネタのよう。
Module 1: Neural Networks
-
Image Classification: Data-driven Approach, k-Nearest Neighbor, train/val/test splits
- L1/L2 distances, hyperparameter search, cross-validation
-
Linear classification: Support Vector Machine, Softmax
- parameteric approach, bias trick, hinge loss, cross-entropy loss, L2 regularization, web demo
-
Optimization: Stochastic Gradient Descent
- optimization landscapes, local search, learning rate, analytic/numerical gradient
-
Backpropagation, Intuitions
- chain rule interpretation, real-valued circuits, patterns in gradient flow
-
Neural Networks Part 1: Setting up the Architecture
- model of a biological neuron, activation functions, neural net architecture, representational power
-
Neural Networks Part 2: Setting up the Data and the Loss
- preprocessing, weight initialization, batch normalization, regularization (L2/dropout), loss functions
-
Neural Networks Part 3: Learning and Evaluation
- gradient checks, sanity checks, babysitting the learning process, momentum (+nesterov), second-order methods, Adagrad/RMSprop, hyperparameter optimization, model ensembles
-
Putting it together: Minimal Neural Network Case Study
- minimal 2D toy data example
Module 2: Convolutional Neural Networks（更新予定）
- Convolutional Neural Networks: Architectures, Convolution / Pooling Layers
- layers, spatial arrangement, layer patterns, layer sizing patterns, AlexNet/ZFNet/VGGNet case studies, computational considerations
- Understanding and Visualizing Convolutional Neural Networks
- tSNE embeddings, deconvnets, data gradients, fooling ConvNets, human comparisons
- Transfer Learning and Fine-tuning Convolutional Neural Networks