[Review] Transition of DRL

Introduction

Agenda

Transition

1. Value Function stands on MDP/POMDP
2. Model Based: Dynamic Programming by Richard Bellman in 1950s.
3. Model Free: TD Learning(Temporal Difference Learning) by Sutton et al., 1988.
4. SARSA/Qlearning introduced by Sutton and Watkins in 1992.
5. Multi-step Bootstrapping: Introduction of $\lambda$ in TD/Q learning.
6. Evolution from Tabular System to large continuous space of environment, which is Function Approximation.
7. Linear Function Approximation EX: SARSA with Linear Function Approximation
8. Neural Function Approximation with RL by Bertsekas et al., 1996