0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

数式で書き下す Local response normalization の Backward propagation

0
Last updated at Posted at 2016-08-10

CNNのLRN(Local response normalization layer)のBackward propagationを実装したときのメモ。

Forward propagationの式は以下。

y_{ ijk }=x_{ ijk }(\kappa +\frac { \alpha  }{ n } \sum _{ t\in G(k) }{ x^2_{ijt} } )^{-\beta}

ここで、$G(k)$ は出力チャンネル $k$ に対応する入力チャンネルの集合とする。

ここで、

C_{ijk} = \kappa+\frac{\alpha}{n}\sum_{t \in G(k)}{x^2_{ijt}} 

とおくと、Forward propagationの式はこうなる。

y_{ijk} = x_{ijk}C_{ijk}^{-\beta}

Backward propagationは、微分の連鎖則より、

\begin{eqnarray*}
\frac{\partial E}{\partial x_{ijd}} 

&=&

\sum_{k:d \in G(k)}{
\frac{\partial E}{\partial y_{ijk}}\frac{\partial y_{ijk}}{\partial x_{ijk}}
} \\

&=&

\frac{\partial E}{\partial y_{ijd}} \frac{\partial y_{ijd}}{\partial x_{ijd}} +
\sum_{k:d \subset G(k)}{\frac{\partial E}{\partial y_{ijk}}\frac{\partial y_{ijk}}{\partial x_{ijk}}} \\

&=&
 
\frac{\partial E}{\partial y_{ijd}}(C_{ijd}^{-\beta}-\frac{2\alpha\beta}{n}x_{ijd}^2C_{ijd}^{-\beta-1}) +
\sum_{k:d\subset G(k)} \frac{\partial E}{\partial y_{ijk}}(-\frac{2\alpha\beta}{n}x_{ijd}x_{ijk}C_{ijk}^{-\beta -1}) \\

&=& 

\frac{\partial E}{\partial y_{ijd}}C_{ijd}^{-\beta} -
\frac{2\alpha\beta}{n}x_{ijd}
\sum_{k:d \in G(k)} 
\frac{\partial E }{\partial y_{ijk}}x_{ijk}C_{ijk}^{-\beta -1} \\

&=&

\frac{\partial E}{\partial y_{ijd}}
(\kappa + \frac{\alpha}{n} \sum_{t \in G(d)}{x_{ijt}^{2}})^{-\beta}
-
\frac{2\alpha\beta}{n}x_{ijd}
\sum_{k:d \in G(k)}
\frac{\partial E}{\partial y_{ijk}}x_{ijk}
(\kappa + \frac{\alpha}{n}\sum_{t \in G(k)}{x_{ijt}^2)^{-\beta -1}}


\end{eqnarray*}

こうなる。

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?