Edited at

AutoAIでお手軽機械学習(その3) Webサービス編


はじめに

AutoAIでお手軽機械学習(その1) 準備編

AutoAIでお手軽機械学習(その2) モデル構築編

の続編です。第三部では「モデル構築編」で作成した機械学習モデルをWatson Machine Learing上のWebサービスとしてデプロイし、呼出しテストを行ってみます。


デプロイ手順

最初にモデルのデプロイ手順を説明します。


プロジェクト管理画面の表示

最初に次のURLを指定して、Watson Studioにログインします。

https://dataplatform.cloud.ibm.com

ログイン後に、作成済みのプロジェクト管理画面を表示します。プロジェクトの選択手順は「その2」に記載しましたので、わからない方はそちらを参照してください。

左から2つめのAssetsタブに、AutoAI Experimentsという欄があるはずです。その中で「その2」で作成したモデルBank Marketingのリンクをクリックします。

詳細画面の下の方に次のようなリストが出て来るので、一番精度の高い、一番上のモデルの「Save as model」リンクをクリックします。

その先のSave as modelパネルでは、Model nameを短めの「Bank Marketing Model」に変更し、画面右下の「Save」ボタンをクリックします。

保存に成功して下のようなパネルが出てきたら、View in projectのリンクをクリックしてください。

下のようなモデル管理画面に遷移します。ここで、右から2つめのDeplymentタブをクリックします。

更に、画面右上のAdd Dploymentリンクをクリックします。

Deploymentの名称を聞かれるので、Bank Marketing Deploymentなどと入力し、画面右下のSaveボタンをクリックしてください。

下のような画面に遷移します。ステータスを確認するため、時々画面をrefreshしてください。

下の画面のように、statusがreadyに遷移したら、Webサービスのデプロイは完了しています。

Bank Marketing Deploymentのリンクをクリックして引き続き、今作ったモデルのテストを行います。


【テスト1】簡易テスト機能から項目値を入力してテストする

下の画面が表示されたら、一番右のDeploymentsタブをクリックします。

(ちなみに、Watson Studioのプロジェクト管理画面から、この画面にアクセスするのは、プロジェクト管理の中のDeploymentsタブをクリックして、対象のDeploymentを選択します。)

下の画面が表示されたら、テスト用の入力値を順に入れて、最後にpredictボタンをクリックします。

入力する値としては、例えば下のテスト用データの値を参考にしてください。


y = yes のテストデータ

項目名

age
38

job
housemaid

marital
divorced

education
secondary

default
no

balance
70

housing
no

loan
no

contact
cellular

day
27

month
oct

duration
255

campaign
3

pdays
148

previous
1

poutcome
success


y = noのテストデータ

項目名

age
30

job
unemployed

marital
married

education
primary

default
no

balance
1787

housing
no

loan
no

contact
cellular

day
19

month
oct

duration
79

campaign
1

pdays
-1

previous
0

poutcome
unknown

呼出しに成功すると、次のような画面になります。

(画面の例では、正しい分類結果となっています。)


【テスト2】簡易テスト機能からjsonで一括テスト

上の方法でテストするのは、今回のモデルのように入力値が多い場合に煩雑です。このような場合、json形式データで複数のテストを同時に行うことができますので、今度はそのケースを試してみます。

下の画面で、「Provide input data as JSON」のアイコンをクリックしてください。

JSONデータのサンプルは結果がyesのケース、noのケースそれぞれについて、テストデータから20件分とってきたものを下記に張り付けておきましたので、こちらをコピペして利用します。


y = yesのテストデータ

{ "input_data":[

{ "fields": ["age","job","marital","education","default","balance","housing","loan",
"contact","day","month","duration","campaign","pdays","previous","poutcome"],
"values": [
[38,"housemaid","divorced","secondary","no",70,"no","no","cellular",27,"oct",255,3,148,1,"success"],
[46,"blue-collar","married","unknown","no",9328,"yes","no","cellular",5,"may",725,3,-1,0,"unknown"],
[52,"admin.","single","unknown","no",2227,"no","no","cellular",24,"feb",242,2,-1,0,"unknown"],
[53,"entrepreneur","married","primary","no",27,"yes","no","telephone",10,"sep",230,3,-1,0,"unknown"],
[28,"blue-collar","married","secondary","no",2788,"yes","no","unknown",6,"jun",369,2,-1,0,"unknown"],
[32,"blue-collar","married","secondary","yes",-53,"yes","no","cellular",16,"apr",648,1,272,1,"other"],
[30,"management","married","tertiary","no",944,"no","no","cellular",29,"oct",251,2,-1,0,"unknown"],
[69,"retired","single","tertiary","no",2144,"no","no","cellular",29,"jul",417,1,184,4,"success"],
[84,"retired","divorced","primary","no",639,"no","no","telephone",18,"may",353,3,-1,0,"unknown"],
[25,"student","single","secondary","no",8,"no","no","cellular",22,"apr",107,2,-1,0,"unknown"],
[47,"blue-collar","divorced","primary","no",126,"yes","no","unknown",3,"jun",2456,2,-1,0,"unknown"],
[50,"self-employed","divorced","tertiary","no",13,"yes","no","cellular",20,"apr",76,1,-1,0,"unknown"],
[30,"technician","married","secondary","no",-393,"yes","yes","cellular",16,"jul",435,1,-1,0,"unknown"],
[55,"entrepreneur","divorced","secondary","no",538,"yes","no","cellular",7,"aug",353,2,-1,0,"unknown"],
[53,"unemployed","married","secondary","no",94,"no","no","cellular",10,"aug",493,1,181,14,"success"],
[57,"retired","married","secondary","no",1044,"no","no","telephone",11,"jan",585,1,192,2,"success"],
[32,"blue-collar","married","primary","no",415,"yes","no","unknown",2,"jun",777,4,-1,0,"unknown"],
[32,"services","single","secondary","no",133,"no","no","cellular",15,"apr",171,1,-1,0,"unknown"],
[48,"blue-collar","married","secondary","no",480,"yes","no","cellular",15,"may",830,1,352,2,"failure"],
[34,"management","married","tertiary","no",606,"no","no","cellular",20,"apr",97,1,-1,0,"unknown"]
]}
]}


y = noのテストデータ

{ "input_data": [

{ "fields": ["age","job","marital","education","default","balance","housing","loan",
"contact","day","month","duration","campaign","pdays","previous","poutcome"],
"values": [
[53,"admin.","divorced","secondary","no",26,"yes","no","cellular",7,"may",56,1,359,1,"failure"],
[36,"technician","married","secondary","no",191,"no","no","cellular",31,"aug",69,1,-1,0,"unknown"],
[58,"technician","divorced","secondary","no",-123,"no","no","cellular",5,"aug",131,2,-1,0,"unknown"],
[26,"student","single","secondary","no",-147,"no","no","unknown",4,"jun",95,2,-1,0,"unknown"],
[34,"technician","single","secondary","no",179,"no","no","cellular",19,"aug",294,3,-1,0,"unknown"],
[55,"blue-collar","married","primary","no",1086,"yes","no","cellular",6,"may",146,1,272,2,"failure"],
[55,"blue-collar","married","secondary","no",471,"yes","no","unknown",30,"may",58,2,-1,0,"unknown"],
[34,"entrepreneur","married","tertiary","no",105,"yes","no","unknown",28,"may",249,2,-1,0,"unknown"],
[41,"entrepreneur","divorced","secondary","no",1588,"yes","yes","unknown",20,"jun",10,8,-1,0,"unknown"],
[28,"blue-collar","single","secondary","no",2700,"no","no","cellular",3,"feb",337,4,-1,0,"unknown"],
[41,"technician","married","secondary","no",1602,"yes","no","cellular",2,"jun",142,1,-1,0,"unknown"],
[30,"admin.","married","tertiary","no",0,"yes","no","cellular",24,"jul",209,2,-1,0,"unknown"],
[39,"technician","married","unknown","no",839,"no","no","unknown",30,"may",825,2,-1,0,"unknown"],
[41,"admin.","married","secondary","no",21515,"yes","no","unknown",5,"jun",87,1,-1,0,"unknown"],
[36,"technician","married","secondary","no",5902,"yes","no","unknown",23,"may",219,4,-1,0,"unknown"],
[37,"management","married","tertiary","no",547,"yes","yes","unknown",21,"may",143,1,-1,0,"unknown"],
[59,"self-employed","married","tertiary","no",3800,"no","yes","cellular",9,"jul",70,1,-1,0,"unknown"],
[26,"technician","single","secondary","no",261,"yes","no","unknown",27,"may",371,2,-1,0,"unknown"],
[43,"technician","married","secondary","no",0,"yes","yes","cellular",4,"feb",539,8,204,5,"failure"],
[42,"management","married","secondary","yes",-34,"no","no","cellular",5,"feb",176,1,-1,0,"unknown"]
]}
]}

呼出し結果例を以下に示します。今回作ったモデルでは正解がyesのデータ20件に対して正解となったのは6件でした。


【テスト3】Watson StudioのJupyter NotebookからAPIを利用して一括テスト

最後に、アプリケーションからAPIを使ってWebサービスを呼び出してみます。

シナリオとして、テスト用にリザーブしてある451件全件のデータに対する分類結果を、Watson StudioのJupyter Notebook上のPython から取得してみます。

Notebookのソース全体のリンクはこちら

https://github.com/makaishi2/sample-data/blob/master/notebooks/bank-marketing-autoai-test.ipynb

Watson Studioのプロジェクト管理画面から先ほどと同じようにAdd to projectをクリックします。

下の画面で今度は、Notebookを選択します。

下の画面で以下の入力をします。

From URLタブをクリック

Nameの欄に Bank-marketing-autoai-testなどの名称を入力

Notebook URLの欄に下記のURLを入力

Create Notebookボタンをクリック

入力するURL:

https://raw.githubusercontent.com/makaishi2/sample-data/master/notebooks/bank-marketing-autoai-test.ipynb

下の画面でWatson Machine Learning認証情報の設定APIのScoring URL設定の2つのセルを設定します。

設定後は shift+enterでセルを順に実行してください。うまくいくと、451件の全テストデータに対して、モデルの予測結果がわかる形になります。

最後に、Notebook上で表示されるはずのグラフサンプルを紹介します。


ROC AUC曲線


モデルを使った場合と使わない場合の成約率の比較