Neuron

\begin{eqnarray}
{\bf u} &=& {\bf W}\cdot {\bf x} \\ 
{\bf z} &=& {\bf f}({\bf u})
\end{eqnarray}

memory cell

\begin{eqnarray}
{\bf s}^t &=& {\bf g}^{Forget,t} \odot {\bf s}^{t-1} + {\bf g}^{Input,t} \odot {\bf f}({\bf u}^{t})\\ 
{\bf u}^t &=& 
 {\bf W}^{in} \cdot {\bf x}^t 
+{\bf W} \cdot {\bf z}^{t-1}  \\
\end{eqnarray}

ここで、$\odot$は要素同士の積を表す。内積でもテンソル積でもなくて、ただのベクトルの要素同士の積。

${\bf f}$は活性化関数。

忘却ゲート ${\bf g}^{Forget,t}$

\begin{eqnarray}
{\bf g}^{Forget,t} &=& {\bf f}({\bf u}^{Forget, t}) \\
 &=& {\bf f}\left( 
  {\bf W}^{Forget,in} \cdot {\bf x}^{t} 
+ {\bf W}^{Forget} \cdot {\bf z}^{t-1} 
+ {\bf w}^{Forget} \odot {\bf s}^{t-1} 
\right)
\end{eqnarray}

入力ゲート ${\bf g}^{Input,t}$

\begin{eqnarray}
{\bf g}^{Input,t} &=& {\bf f}({\bf u}^{Input, t}) \\
 &=& {\bf f}\left( 
  {\bf W}^{Input,in} \cdot {\bf x}^{t} 
+ {\bf W}^{Input} \cdot {\bf z}^{t-1} 
+ {\bf w}^{Input} \odot {\bf s}^{t-1} 
\right)
\end{eqnarray}

memory unitからの出力

\begin{eqnarray}
{\bf z}^{t} &=& {\bf g}^{Output, t} \odot
{\bf f}({\bf s}^{t})
\end{eqnarray}

出力ゲート ${\bf g}^{Output,t}$

\begin{eqnarray}
{\bf g}^{Output,t} &=& {\bf f}({\bf u}^{Output, t}) \\
 &=& {\bf f}\left( 
  {\bf W}^{Output,in} \cdot {\bf x}^{t} 
+ {\bf W}^{Output} \cdot {\bf z}^{t-1} 
+ {\bf w}^{Output} \odot {\bf s}^{t} 
\right)
\end{eqnarray}