Introduction
Hello everyone! It's been almost more than a few decades since the theoretical importance of DL was academically proposed.
Needless to say, but I have been studying theoretical aspects of DL long. But when it comes to deep understanding, we cannot avoid the actual experiment.
So, these days Google or Apple or other IT giants put more efforts for contributing the open source DL libraries enhancing the development.
Hence, I would like to study and review keras APIs to show the sample usage of them in this article.
Requirements
keras: 2.2.0 as of 2018/6/14
Python: 3.6
Sequential model link:
https://keras.io/getting-started/sequential-model-guide/
Architectures
1. LSTM
Basic one
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.layers import Embedding
from keras.layers import LSTM
max_features = 20000
maxlen = 80
batch_size = 32
print('Loading data...')
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
print(len(x_train), 'train sequences')
print(len(x_test), 'test sequences')
print('Pad sequences (samples x time)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen).reshape((x_train.shape[0], maxlen))
x_test = sequence.pad_sequences(x_test, maxlen=maxlen).reshape((x_test.shape[0], maxlen))
print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)
model = Sequential()
model.add(Embedding(max_features, output_dim=256))
model.add(LSTM(128))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.fit(x_train, y_train, batch_size=16, epochs=10)
score = model.evaluate(x_test, y_test, batch_size=16)
Key APIs
sequence.pad_sequences
Arguments
sequences: List of lists, where each element is a sequence.
maxlen: Int, maximum length of all sequences.
dtype: Type of the output sequences.
padding: String, 'pre' or 'post': pad either before or after each sequence.
truncating: String, 'pre' or 'post': remove values from sequences larger than maxlen, either at the beginning or at the end of the sequences.
value: Float, padding value.
Returns
x: Numpy array with shape (len(sequences), maxlen)
example
import numpy as np
a = np.random.randn(2, 10)
sequence.pad_sequences(a, maxlen=12, value=0, dtype='float32')
## output
array([[ 0. , 0. , -1.6152234 , -0.59010035, -0.35678408,
-2.4155464 , -0.08363751, -0.9100027 , -0.03029301, 0.6607818 ,
-0.40585187, -0.2225399 ],
[ 0. , 0. , 2.6790402 , -0.44190383, -1.7504224 ,
-3.3369403 , 0.612109 , 1.8488977 , -1.2793024 , -0.75258255,
-1.2619722 , -0.28750297]], dtype=float32)
Model Class API
Arguments
optimizer: String (name of optimizer) or optimizer instance. See optimizers.
loss: String (name of objective function) or objective function. See losses. If the model has multiple outputs, you can use a different loss on each output by passing a dictionary or a list of losses. The loss value that will be minimized by the model will then be the sum of all individual losses.
metrics: List of metrics to be evaluated by the model during training and testing. Typically you will use metrics=['accuracy']. To specify different metrics for different outputs of a multi-output model, you could also pass a dictionary, such as metrics={'output_a': 'accuracy'}.
loss_weights: Optional list or dictionary specifying scalar coefficients (Python floats) to weight the loss contributions of different model outputs. The loss value that will be minimized by the model will then be the weighted sum of all individual losses, weighted by the loss_weights coefficients. If a list, it is expected to have a 1:1 mapping to the model's outputs. If a tensor, it is expected to map output names (strings) to scalar coefficients.
sample_weight_mode: If you need to do timestep-wise sample weighting (2D weights), set this to "temporal". None defaults to sample-wise weights (1D). If the model has multiple outputs, you can use a different sample_weight_mode on each output by passing a dictionary or a list of modes.
weighted_metrics: List of metrics to be evaluated and weighted by sample_weight or class_weight during training and testing.
target_tensors: By default, Keras will create placeholders for the model's target, which will be fed with the target data during training. If instead you would like to use your own target tensors (in turn, Keras will not expect external Numpy data for these targets at training time), you can specify them via the target_tensors argument. It can be a single tensor (for a single-output model), a list of tensors, or a dict mapping output names to target tensors.
**kwargs: When using the Theano/CNTK backends, these arguments are passed into K.function. When using the TensorFlow backend, these arguments are passed into tf.Session.run.
Pythonic one
from keras.models import Model
from keras.layers import Dense, Dropout, Embedding, LSTM, Input
from keras.datasets import imdb
from keras.preprocessing import sequence
max_features = 20000
maxlen = 80
batch_size = 32
print('Loading data...')
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
print(len(x_train), 'train sequences')
print(len(x_test), 'test sequences')
print('Pad sequences (samples x time)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen).reshape((x_train.shape[0], maxlen))
x_test = sequence.pad_sequences(x_test, maxlen=maxlen).reshape((x_test.shape[0], maxlen))
print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)
inputs = Input(shape=(maxlen, ))
x = Embedding(max_features, output_dim=256)(inputs)
x = LSTM(128)(x)
x = Dropout(0.5)(x)
outputs = Dense(1, activation='sigmoid')(x)
model = Model(input=inputs, output=outputs)
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
model.fit(x_train, y_train, batch_size=16, epochs=10)
score = model.evaluate(x_test, y_test, batch_size=16)
then, we can certainly play with the output of each layer!
CNN, MLP
I am using a mnist Dataset.
import keras
from keras.datasets import mnist
from keras.layers import Input, Dense
from keras.models import Model
def show_image(index):
import matplotlib.pyplot as plt
plt.imshow(x_train[index])
plt.show()
def check_index(a):
count = 0
for i in a.reshape(a.shape[1]):
if i == 1:
return count
else:
count += 1
batch_size = 128
num_classes = 10
epochs = 2
rows, cols = 28, 28
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train_ = x_train.reshape(x_train.shape[0], rows*cols)
x_test_ = x_test.reshape(x_test.shape[0], rows*cols)
x_train_ = x_train_.astype('float32')
x_test_ = x_test_.astype('float32')
x_train_ /= 255
x_test_ /= 255
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
inputs = Input(shape=(784, ), name='input')
x = Dense(64, activation='relu', name='l1')(inputs)
x = Dense(64, activation='relu', name='l2')(x)
predictions = Dense(10 ,activation='softmax', name='output')(x)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train_, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_test_, y_test))
score = model.evaluate(x_test_, y_test, verbose=0)
print("Loss: {0}, accuracy: {1}".format(score[0], score[1]))
sample = x_train[12].reshape(1, rows*cols)
pred = model.predict(sample)
print("prediction: ", check_index(pred))
show_image(12)
from __future__ import print_function
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K
def show_image(index):
import matplotlib.pyplot as plt
plt.imshow(x_train[index])
plt.show()
def check_index(a):
count = 0
for i in a.reshape(a.shape[1]):
if i == 1:
return count
else:
count += 1
batch_size = 128
num_classes = 10
epochs = 2
# input image dimensions
img_rows, img_cols = 28, 28
# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
if K.image_data_format() == 'channels_first':
x_train_ = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)
x_test_ = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)
input_shape = (1, img_rows, img_cols)
else:
x_train_ = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)
x_test_ = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)
input_shape = (img_rows, img_cols, 1)
x_train_ = x_train_.astype('float32')
x_test_ = x_test_.astype('float32')
x_train_ /= 255
x_test_ /= 255
print('x_train_ shape:', x_train_.shape)
print(x_train_.shape[0], 'train samples')
print(x_test_.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
model.fit(x_train_, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test_, y_test))
score = model.evaluate(x_test_, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
sample = x_train[12].reshape(1, rows, cols, 1)
pred = model.predict(sample)
print("prediction: ", check_index(pred))
show_image(12)