Neuron
\begin{eqnarray}
{\bf u} &=& {\bf W}\cdot {\bf x} \\
{\bf z} &=& {\bf f}({\bf u})
\end{eqnarray}
memory cell
\begin{eqnarray}
{\bf s}^t &=& {\bf g}^{Forget,t} \odot {\bf s}^{t-1} + {\bf g}^{Input,t} \odot {\bf f}({\bf u}^{t})\\
{\bf u}^t &=&
{\bf W}^{in} \cdot {\bf x}^t
+{\bf W} \cdot {\bf z}^{t-1} \\
\end{eqnarray}
ここで、$\odot$は要素同士の積を表す。内積でもテンソル積でもなくて、ただのベクトルの要素同士の積。
${\bf f}$は 活性化関数。
忘却ゲート ${\bf g}^{Forget,t}$
\begin{eqnarray}
{\bf g}^{Forget,t} &=& {\bf f}({\bf u}^{Forget, t}) \\
&=& {\bf f}\left(
{\bf W}^{Forget,in} \cdot {\bf x}^{t}
+ {\bf W}^{Forget} \cdot {\bf z}^{t-1}
+ {\bf w}^{Forget} \odot {\bf s}^{t-1}
\right)
\end{eqnarray}
入力ゲート ${\bf g}^{Input,t}$
\begin{eqnarray}
{\bf g}^{Input,t} &=& {\bf f}({\bf u}^{Input, t}) \\
&=& {\bf f}\left(
{\bf W}^{Input,in} \cdot {\bf x}^{t}
+ {\bf W}^{Input} \cdot {\bf z}^{t-1}
+ {\bf w}^{Input} \odot {\bf s}^{t-1}
\right)
\end{eqnarray}
memory unitからの出力
\begin{eqnarray}
{\bf z}^{t} &=& {\bf g}^{Output, t} \odot
{\bf f}({\bf s}^{t})
\end{eqnarray}
出力ゲート ${\bf g}^{Output,t}$
\begin{eqnarray}
{\bf g}^{Output,t} &=& {\bf f}({\bf u}^{Output, t}) \\
&=& {\bf f}\left(
{\bf W}^{Output,in} \cdot {\bf x}^{t}
+ {\bf W}^{Output} \cdot {\bf z}^{t-1}
+ {\bf w}^{Output} \odot {\bf s}^{t}
\right)
\end{eqnarray}