Help us understand the problem. What is going on with this article?

Core MLモデルの入力の型をMLMultiArrayから画像(CVPixelBuffer)に変更する

次のように普通にKerasでMNISTモデルをつくってcoremltoolsでCore MLモデルに変換すると、入力の型がmultiArrayType(Core MLではMLMultiArray)になる。

(Kerasでモデル構築)

def create_keras_base_model(url):    
    import keras
    from keras.models import Sequential
    from keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPooling2D

    keras.backend.clear_session()
    model = Sequential()
    model.add(Conv2D(32, kernel_size=(3, 3),
                     activation='relu',
                     input_shape=(28, 28, 1)))
    model.add(Conv2D(64, (3, 3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))
    model.add(Flatten())
    model.add(Dense(128, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(10, activation='softmax'))

    model.compile(loss=keras.losses.categorical_crossentropy,
                  optimizer=keras.optimizers.SGD(lr=0.01),
                  metrics=['accuracy'])

    model.save(url)

keras_model_path = './KerasMnist.h5'
create_keras_base_model(keras_model_path)

(coremltoolsで.mlmodelファイルに変換)

def convert_keras_to_mlmodel(keras_url, mlmodel_url):
    from keras.models import load_model
    keras_model = load_model(keras_url)

    from coremltools.converters import keras as keras_converter
    class_labels = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']
    mlmodel = keras_converter.convert(keras_model, input_names=['image'],
                                output_names=['digitProbabilities'],
                                class_labels=class_labels,
                                predicted_feature_name='digit')

    mlmodel.save(mlmodel_url)

coreml_model_path = './MNISTDigitClassifier.mlmodel'
convert_keras_to_mlmodel(keras_model_path , coreml_model_path)

(作成したモデルの入力形式を確認する)

import coremltools
spec = coremltools.utils.load_spec(coreml_model_path)
builder = coremltools.models.neural_network.NeuralNetworkBuilder(spec=spec)

builder.inspect_input_features()

出力結果:

[Id: 0] Name: image
          Type: multiArrayType {
  shape: 1
  shape: 28
  shape: 28
  dataType: DOUBLE
}

形状が1x28x28、値がDOUBLE型のmultiArrayTypeということで、このままだといろいろとめんどくさい

たとえば入力型が画像じゃないと、Visionで扱えないので、Core ML単体で扱うことになる。

Visionなしで、Core ML単体で使う - Qiita

MLMultiArrayの扱いは慣れてないと色々と戸惑うかもしれない。

Core MLのモデルの出力がMLMultiArrayの場合のメモ|shu223|note

というわけで、画像型に変更する方法が以下:

coremltoolsでmlmodelの入力の型を変更する

(入力の型を1x28x28のmulti arrayから28x28のグレースケール画像に変更)

neuralnetwork_spec = builder.spec

# change the input so the model can accept 28x28 grayscale images
neuralnetwork_spec.description.input[0].type.imageType.width = 28
neuralnetwork_spec.description.input[0].type.imageType.height = 28

from coremltools.proto import FeatureTypes_pb2 as _FeatureTypes_pb2
grayscale = _FeatureTypes_pb2.ImageFeatureType.ColorSpace.Value('GRAYSCALE')
neuralnetwork_spec.description.input[0].type.imageType.colorSpace = grayscale

(確認)

builder.inspect_input_features()

出力結果:

[Id: 0] Name: image
          Type: imageType {
  width: 28
  height: 28
  colorSpace: GRAYSCALE
}

.mlmodelファイルをXcodeで見てみると、意図した通りになっている。

Screen Shot 2019-11-18 at 9.25.43.png

自動生成される入力クラスを見てみると、入力の型がCVPixelBufferになっている。

var image: CVPixelBuffer

これでVisionでお手軽に扱える。

参考

全面的にcoremltools公式リポジトリのexamplesにあるサンプルを参考にした。

https://github.com/apple/coremltools/tree/master/examples/updatable_models

Why do not you register as a user and use Qiita more conveniently?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away