Help us understand the problem. What is going on with this article?

『ゼロから作るDeep Learning』5章 メモ

More than 1 year has passed since last update.

斎藤康毅:『ゼロから作るDeep Learning』オライリージャパン,2016.
理解を深めるためにメモします.

5章 誤差逆伝播法

誤差逆伝播法
学習の際,重みパラメータに関する損失関数の勾配を数値微分によって求めたが,数値微分より,効率よく行う手法

導入される数学の概念

  • 計算グラフ (computational graph)
  • 順伝播 (forward propagation)
  • 逆伝播 (backward propagation)
  • アフィン (Affine) 変換
    幾何学の分野で,ニューラルネットワークの順伝播で行う行列の内積のこと

式 $(5.12)$ を丁寧に書く.

まず,
$
\begin{eqnarray}
y
&=&
\dfrac {1} { 1+\exp \left( -x\right)}
\tag{5.9}
\end{eqnarray}
$

なので,式 $(5.9)$ より
$$
y^{2}=
\dfrac {1^{2}} {\left( 1+\exp \left( -x\right) \right) ^{2}}
$$

を準備しておく.
式 $(5.12)$ を解いていく.

\begin{eqnarray}
\dfrac {\partial L} {dy}y^{2}\exp \left( -x\right)
&=& 
\dfrac {\partial L} {dy}\dfrac {1} {\left( 1+\exp \left( -x\right) \right) ^{2}}\exp \left( -x\right)\\
&=& 
\dfrac {\partial L} {dy}\dfrac {1^{2}} {\left( 1+\exp \left( -x\right) \right)}\dfrac{\exp \left( -x\right)}{\left( 1+\exp \left( -x\right) \right)}\\
&=& 
\dfrac {\partial L} {dy}y\dfrac{\exp \left( -x\right)}{ 1+\exp \left( -x\right)}\\
&=& 
\dfrac {\partial L} {dy}y\left(\dfrac {\exp \left( -x\right)} { 1+\exp \left( -x\right)}+\dfrac {1} { 1+\exp \left( -x\right)}-\dfrac {1} { 1+\exp \left( -x\right)}\right)\\
&=& 
\dfrac {\partial L} {dy}y\left(\dfrac {1+\exp \left( -x\right)} { 1+\exp \left( -x\right)}-\dfrac {1} { 1+\exp \left( -x\right)}\right)\\
&=& 
\dfrac {\partial L} {dy}y\left(1-\dfrac {1} { 1+\exp \left( -x\right)}\right)\\
&=& 
\dfrac {\partial L} {dy}y(1-y)\\
\end{eqnarray}

間違い等あれば教えていただけると嬉しいです.

随時更新するかも :snowflake:

Why do not you register as a user and use Qiita more conveniently?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away