CD法メモ

Gibbs distribution

\begin{eqnarray}
p(\bf{x}; \bf{\xi}) &=& \frac{e^{-E(\bf{x}; \bf{\xi})}}{Z}  \\
Z &=& \sum_{\bf{x}} e^{-E(\bf{x}; \bf{\xi})}
\end{eqnarray}

likelihood

\begin{eqnarray}
\frac{\partial}{\partial \bf{\xi}} \log \left( p(\bf{x}; \bf{\xi})\right) 
 &=& 
 \frac{\partial}{\partial \bf{\xi}} \log 
  \left( 
   \frac{e^{-E(\bf{x}; \bf{\xi})}}{Z}
  \right) \\
 &=& 
  - \frac{\partial}{\partial \bf{\xi}} 
   E(\bf{x}; \bf{\xi})
  - \frac{\partial}{\partial \bf{\xi}} \log(Z) \\
 &=& 
  - \frac{\partial}{\partial \bf{\xi}} 
   E(\bf{x}; \bf{\xi})
  - \frac{\partial}{\partial \bf{\xi}} 
   \log
    \left(
      \sum_{\bf{x}} e^{-E(\bf{x}; \bf{\xi})} 
    \right) \\
 &=&
  - \frac{\partial}{\partial \bf{\xi}} 
   E(\bf{x}; \bf{\xi})
  - \frac{1}{Z}
      \sum_{\bf{x}} 
     \frac{\partial}{\partial \bf{\xi}} \left(e^{-E(\bf{x}; \bf{\xi})} \right) \\
 &=&
  - \frac{\partial}{\partial \bf{\xi}} 
   E(\bf{x}; \bf{\xi})
  - \sum_{\bf{x}} 
   \frac{e^{-E(\bf{x}; \bf{\xi})}}{Z}
     \frac{\partial}{\partial \bf{\xi}} \left( -E(\bf{x}; \bf{\xi}) \right) \\
 &=&
  - \frac{\partial}{\partial \bf{\xi}} 
   E(\bf{x}; \bf{\xi})
  +  \left \langle
      \frac{\partial}{\partial \bf{\xi}} E(\bf{x}; \bf{\xi}) 
     \right \rangle_{\bf{x}} \\
\end{eqnarray}

Contrastive Divergence method

• replace the second term sampling summations
• it is known that a few sampling trial works well empirically.