LoginSignup
6

More than 5 years have passed since last update.

スパース自己符号化器

Last updated at Posted at 2015-12-15

スパース自己符号化器を使えば中間層のユニット数の方が多い場合でも自己符号化器で
意味のある表現を学習ができる。
chainerのサンプルコートmnistで実装するとこうなる。。。

n_units=1000
model = FunctionSet(l1=F.Linear(784, n_units),
                    l2=F.Linear(n_units, 784))

sigma0 = 0.05
beta = 0.1

def forward(x_data,y_data):
    x, t = Variable(x_data), Variable(y_data)
    y = F.sigmoid(model.l1(x))
    x_hat  = model.l2(y)
    loss = F.mean_squared_error(x_hat, t)
    A_data = np.matrix(np.repeat(1.0/batchsize,batchsize).astype(np.float32))
    A = Variable(A_data)
    y_avg = F.matmul(A,y)
    kl = sigma0 * np.log(sigma0) - sigma0 * F.sum(F.log(y_avg)) \
          + (1-sigma0) * np.log(1-sigma0) - (1-sigma0) * F.sum(F.log(1 - y_avg))
    loss = loss + beta * kl

    return loss

次回mnistで検証してみよう。

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
6