Help us understand the problem. What is going on with this article?

ゼロから作るDeepLearning 様々なレイヤの実装

学習内容の目次

誤差逆伝播法

  • 様々なレイヤの実装

要約

☆誤差逆伝播法

  • 前回,重みパラメータを求めるための損失関数は数値微分で求めたがこの方法では計算に時間がかかるため,今回は誤差逆伝播法を使う
  • 逆伝播法とは名前の通り伝播法(左から右へと順番に計算処理を行う)の逆で計算を行う方法
  • 数値微分でパラメータを求めるよりも計算が高速

☆様々なレイヤの実装

  • レイヤには多数の種類があり,これを用いてニューラルネットワークを構成する↓

◯乗算レイヤの実装

In[1]:
class MulLayer:
    def __init__(self):
        self.x = None
        self.y = None

    # 順伝播
    def forward(self, x, y):
        self.x = x
        self.y =y
        out = x * y

        return out

    # 逆伝播
    def backward(self, dout):
        # xとyをひっくり返す
        dx = dout * self.y
        dy = dout * self.x

        return dx, dy        

◯加算レイヤの実装

In[2]:
# 加算レイヤの実装
class AddLayer:
    def __init__(self):
        pass

    # 順伝播
    def forward(self, x, y):
        out = x + y
        return out

    # 逆伝播
    def backward(self, dout):
        dx = dout * 1
        dy = dout * 1
        return dx, dy

◯ReLuレイヤの実装

In[3]:
# Reluレイヤの実装
class ReLu:
    def __init__(self):
        self.mask = None

    # maskはxが0以下の場所をTrue,0以上のbashowoFalseとして保存する
    def forward(self, x):
        self.mask = (x <= 0)
        out = x.copy()
        out[self.mask] = 0

        return out

    def backward(self, dout):
        dout[self.mask] = 0
        dx = dout

        return dx

◯Sigmoidレイヤの実装

In[4]:
# Sigmoidレイヤ
class Sigmoid:
    def __init__(self):
        self.out = None

    # 順伝播
    # outに出力を保存
    def forward(self, x):
        out = 1 / (1 + np.exp(-x))
        self.out = out

        return out

    #  逆伝播
    def backward(self, dout):
        dx = dout * (1.0 - self.out) * self.out

        return dx

◯Affineレイヤの実装

In[5]:
# Affineレイヤ
class Affine:
    def __init__(self, W, b):
        self.W = W
        self.b = b
        self.x = None
        self.dW = None
        self.db = None

    def forward(self, x):
        self.x = x
        out = np.dot(x, self.W) + self.b

        return out

    def backward(self, dout):
        dx = np.dot(dout, self.W.T)
        self.dW = np.dot(self.x.T, dout)
        self.db = np.sum(dout, axis-0)

        return dx

◯Softmaxレイヤの実装

In[6]:
# Softmaxレイヤ
class SoftmaxWithLoss:
    def __init__(self):
        # 損失
        self.loss = None
        # softmaxの出力
        self.y = None
        # 教師データ
        self.t = None

    def forward(self, x, t):
        self.t = t
        self.y = softmax(x)
        self.loss = cross_entropy_error(self.y, self.t)

        return self.loss

    def backward(self, dout=1):
        batch_size = self.t.shape[0]
        dx = (self.y - self.t) / batch_size

        return dx

ポイント

  • 順伝播を行った後に逆伝播を行う
  • レイヤと計算の流れを図でイメージできていないと理解し難いところなので丁寧に進める!
  • 各レイヤやクラスの役割を明確にする

使用想定場面

  • これらのレイヤを組み合わせることでニュートラルネットワークを構築する!つまり必須
Why do not you register as a user and use Qiita more conveniently?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away