TJBot zeroと音声でやり取りをしたい。そんな人向けの拡張コンテンツです。
TJBot zeroの公式サイトのガイドや、BMXUG主催のWorkshopなどでTJBot zeroを製作した方を対象にしています。
オリジナルのTJBotのNode.jsプログラムでは、WatsonのSpeech to TextとWatsonAssistantそしてText to Speechの組み合わせで会話できる仕組みが作られていますが、会話がない時もずっとSpeech to Textが動き続けているので、あっという間にライトアカウントの無償枠を使い切ってします。（現在は、タイムアウトの設定も追加されています）
そこで、TJBot zreoにプシュスイッチを追加して、ボタンが押された後に3秒間命令を聞きとった後で、命令をWatson Assistantで解釈し、アクションするフローのインポートと、Watson AssitantのSkillのインポート方法などについて解説します。
↑クリックするとYoutubeのビデオが再生されます。
##ハードウェアの準備
TJBot zeroにプッシュスイッチを追加します。
⬇︎今回使用したプッシュスイッチ
⬇︎TJBot zeroの後ろ側にプッシュスイッチをつけた様子
プッシュスイッチには、モーメンタリーとオルタネイトという2種類のスイッチがあり、今回はモーメンタリー・スイッチを使っています。この2種類のスイッチの違いは、モーメンタリーは押している間だけオンになり、離すとオフになります。一方オルタネイトは、一度押すとロックがかかりオンになりっぱなしになり、再度押すとロックが外れオフになるスイッチです。
新しく買う場合は、モーメンタリー・スイッチを選んでください。
##追加しておく機能（ノード）とWatsonの機能
今回のシナリオ（フロー）で特有な機能として、Text to Speechから直接スピーカーを鳴らす、node-red-contrib-speakerpiがありますので、node-red-contrib-speakerpiのページに記載されている下記のコマンドでRaspbery PiのNode-Red用にインストールをしておきます。
$cd ~/.node-red
$sudo npm install node-red-contrib-speakerpi
⬆︎Raspberry Piのコマンドプロンプトから下記のコマンドを使用してnode-red-contrib-speakerpiをインストールする。
今回のフローで使用するWatsonの機能としては、Speech to Text, Text to Speech, Visual Recognition, Assitantなどがあります。Assitantは設定なども必要なので後ほど手順を紹介しますが、それ以外の機能がIBM Cloudのダッシュボード上で使えるようになっていることを確認しておいてください。設定方法は、TJBot zeroの公式サイトにあるインストールガイドをご参照ください。
##フローの読み込みと設定
Raspberry Pi上のNode-Redエディターの画面右上にある三本線メニューから、読み込み=>クリップボードの順でクリックし、下記のフロー・コードをカット・アンド・ペーストして読み込みます。
[{"id":"639f1e20.f58248","type":"subflow","name":"control led","info":"","category":"Raspberry Pi","in":[{"x":40,"y":100,"wires":[{"id":"e1566235.d0bee"},{"id":"5c33f332.aa607c"},{"id":"621f7895.3cca88"}]}],"out":[],"icon":"node-red/rpi.png"},{"id":"a3299c90.98d6c8","type":"rpi-gpio out","z":"639f1e20.f58248","name":"Pin15: red","pin":"15","set":true,"level":"0","freq":"","out":"out","x":410,"y":40,"wires":[]},{"id":"b10d349f.0390b8","type":"rpi-gpio out","z":"639f1e20.f58248","name":"Pin16: green","pin":"16","set":true,"level":"0","freq":"","out":"out","x":410,"y":100,"wires":[]},{"id":"42c7f479.c1cd64","type":"rpi-gpio out","z":"639f1e20.f58248","name":"Pin18: blue","pin":"18","set":true,"level":"0","freq":"","out":"out","x":410,"y":160,"wires":[]},{"id":"e1566235.d0bee","type":"change","z":"639f1e20.f58248","name":"payload = payload.r","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.r","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":200,"y":40,"wires":[["a3299c90.98d6c8"]]},{"id":"5c33f332.aa607c","type":"change","z":"639f1e20.f58248","name":"payload = payload.g","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.g","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":200,"y":100,"wires":[["b10d349f.0390b8"]]},{"id":"621f7895.3cca88","type":"change","z":"639f1e20.f58248","name":"payload = payload.b","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.b","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":200,"y":160,"wires":[["42c7f479.c1cd64"]]},{"id":"cfbd9ed9.13ff4","type":"tab","label":"Assistant","disabled":false,"info":""},{"id":"86ef6c72.9ae678","type":"rpi-gpio in","z":"cfbd9ed9.13ff4","name":"","pin":"22","intype":"up","debounce":"25","read":false,"x":100,"y":80,"wires":[["88a417ca.f82b68"]]},{"id":"6aad6fe8.c2357","type":"watson-speech-to-text","z":"cfbd9ed9.13ff4","name":"Speech to Text","alternatives":1,"speakerlabels":false,"smartformatting":false,"lang":"ja-JP","langhidden":"ja-JP","langcustom":"NoCustomisationSetting","langcustomhidden":"","band":"BroadbandModel","bandhidden":"BroadbandModel","password":"","apikey":"","payload-response":true,"streaming-mode":false,"streaming-mute":true,"auto-connect":false,"discard-listening":false,"disable-precheck":false,"default-endpoint":false,"service-endpoint":"https://gateway-tok.watsonplatform.net/speech-to-text/api","x":660,"y":160,"wires":[["a80882e6.55bd78"]]},{"id":"3b0c77c6.ad0a3","type":"file in","z":"cfbd9ed9.13ff4","name":"Read voice file","filename":"/home/pi/speech.wav","format":"","sendError":true,"x":480,"y":160,"wires":[["6aad6fe8.c2357"]]},{"id":"d58aee22.4c0f88","type":"exec","z":"cfbd9ed9.13ff4","command":"arecord -D plughw:1,0 -f cd -d 3 /home/pi/speech.wav","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"命令録音","x":320,"y":160,"wires":[["3b0c77c6.ad0a3"],[],[]]},{"id":"97f9bbb3.2be49","type":"rpi-gpio out","z":"cfbd9ed9.13ff4","name":"Pin16: 緑","pin":"16","set":true,"level":"0","freq":"","out":"out","x":580,"y":80,"wires":[]},{"id":"89421984.420b28","type":"watson-conversation-v1","z":"cfbd9ed9.13ff4","name":"","workspaceid":"64e1fa15-72de-48a0-ab84-3711442eb93e","multiuser":false,"context":true,"empty-payload":false,"default-endpoint":false,"service-endpoint":"https://gateway-tok.watsonplatform.net/assistant/api","timeout":"","optout-learning":false,"x":440,"y":240,"wires":[["70610577.9b3694","20f1132c.69485c","abf23073.1944b8"]]},{"id":"70610577.9b3694","type":"debug","z":"cfbd9ed9.13ff4","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload.intents[0].intent","x":720,"y":220,"wires":[]},{"id":"20f1132c.69485c","type":"switch","z":"cfbd9ed9.13ff4","name":"","property":"payload.intents[0].intent","propertyType":"msg","rules":[{"t":"eq","v":"raise-arm","vt":"str"},{"t":"eq","v":"lower-arm","vt":"str"},{"t":"eq","v":"wave","vt":"str"},{"t":"eq","v":"shine","vt":"str"},{"t":"eq","v":"see","vt":"str"},{"t":"nnull"}],"checkall":"false","repair":false,"outputs":6,"x":190,"y":360,"wires":[["8405f4a7.ae84b8"],["27ec0f02.f1ee28"],["5c573fdd.9245e8"],["4cc07b97.914de4"],["2cac96d1.7ff742"],["580a027a.b0878c"]]},{"id":"6dd88ced.dbd454","type":"rpi-gpio out","z":"cfbd9ed9.13ff4","name":"Pin26: サーボモーター制御","pin":"26","set":"","level":"0","freq":"50","out":"pwm","x":870,"y":320,"wires":[]},{"id":"7737ebba.79971c","type":"trigger","z":"cfbd9ed9.13ff4","op1":"","op2":"0","op1type":"pay","op2type":"str","duration":"200","extend":false,"units":"ms","reset":"","bytopic":"all","name":"","x":600,"y":320,"wires":[["6dd88ced.dbd454"]]},{"id":"8405f4a7.ae84b8","type":"function","z":"cfbd9ed9.13ff4","name":"手をあげる(2)","func":"msg.payload = 2;\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":320,"wires":[["7737ebba.79971c"]]},{"id":"27ec0f02.f1ee28","type":"function","z":"cfbd9ed9.13ff4","name":"手を下げる(7)","func":"msg.payload = 7;\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":360,"wires":[["a4c268.78a94d98"]]},{"id":"a4c268.78a94d98","type":"trigger","z":"cfbd9ed9.13ff4","op1":"","op2":"0","op1type":"pay","op2type":"str","duration":"200","extend":false,"units":"ms","reset":"","bytopic":"all","name":"","x":600,"y":360,"wires":[["6dd88ced.dbd454"]]},{"id":"580a027a.b0878c","type":"function","z":"cfbd9ed9.13ff4","name":"メッセージ転送","func":"msg.payload = msg.payload.output.text[0];\nreturn msg;","outputs":1,"noerr":0,"x":380,"y":1000,"wires":[["839b33e7.cd55e"]]},{"id":"839b33e7.cd55e","type":"watson-text-to-speech","z":"cfbd9ed9.13ff4","name":"Text to Speech","lang":"ja-JP","langhidden":"ja-JP","langcustomhidden":"","voice":"ja-JP_EmiVoice","voicehidden":"en-US_LisaVoice","format":"audio/wav","password":"","apikey":"","payload-response":false,"default-endpoint":false,"service-endpoint":"https://gateway-tok.watsonplatform.net/text-to-speech/api","x":720,"y":1000,"wires":[["9b98b9a7.03ee68"]]},{"id":"5c573fdd.9245e8","type":"function","z":"cfbd9ed9.13ff4","name":"手を振る(12)","func":"msg.payload = 12;\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":400,"wires":[["6dd88ced.dbd454","d4ae6453.fd30b8"]]},{"id":"272d5d52.c9b2c2","type":"trigger","z":"cfbd9ed9.13ff4","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"400","extend":false,"units":"ms","reset":"","bytopic":"all","name":"","x":920,"y":400,"wires":[["6dd88ced.dbd454"]]},{"id":"d4ae6453.fd30b8","type":"delay","z":"cfbd9ed9.13ff4","name":"","pauseType":"delay","timeout":"400","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":590,"y":400,"wires":[["974f22ef.97dc9"]]},{"id":"974f22ef.97dc9","type":"function","z":"cfbd9ed9.13ff4","name":"手をあげる(2)","func":"msg.payload = 2;\nreturn msg;","outputs":1,"noerr":0,"x":750,"y":400,"wires":[["272d5d52.c9b2c2"]]},{"id":"a80882e6.55bd78","type":"function","z":"cfbd9ed9.13ff4","name":"設定","func":"msg.additional_context = {\"timezone\":\"Asia/Tokyo\"};\nreturn msg;","outputs":1,"noerr":0,"x":310,"y":240,"wires":[["89421984.420b28"]]},{"id":"2cac96d1.7ff742","type":"camerapi-takephoto","z":"cfbd9ed9.13ff4","filemode":"0","filename":"photo1.jpeg","filedefpath":"1","filepath":"","fileformat":"jpeg","resolution":"1","rotation":"0","fliph":"0","flipv":"0","brightness":"50","contrast":"0","sharpness":"0","quality":"80","imageeffect":"none","exposuremode":"auto","iso":"0","agcwait":"1.0","led":"0","awb":"auto","name":"","x":400,"y":760,"wires":[["f4774936.c99e78"]]},{"id":"f4774936.c99e78","type":"visual-recognition-v3","z":"cfbd9ed9.13ff4","name":"","vr-service-endpoint":"https://gateway.watsonplatform.net/visual-recognition/api","image-feature":"classifyImage","lang":"ja","x":590,"y":760,"wires":[["6d36ff2e.c0995"]]},{"id":"6d36ff2e.c0995","type":"function","z":"cfbd9ed9.13ff4","name":"データ抽出","func":"var classes = [];\nmsg.payload = {};\n// レスポンスからclassの配列を取り出す\nclasses = msg.result.images[0].classifiers[0].classes;\nmsg.payload.classes = classes;\nmsg.payload.imageSrc = msg.result.images[0].resolved_url;\nreturn msg;","outputs":1,"noerr":0,"x":770,"y":760,"wires":[["daccfec8.de597"]]},{"id":"daccfec8.de597","type":"switch","z":"cfbd9ed9.13ff4","name":"","property":"payload.classes.length","propertyType":"msg","rules":[{"t":"eq","v":"0","vt":"str"},{"t":"eq","v":"1","vt":"str"},{"t":"eq","v":"2","vt":"str"},{"t":"gt","v":"2","vt":"str"}],"checkall":"false","repair":false,"outputs":4,"x":370,"y":860,"wires":[["432d3fc.4cce44"],["cd40e2ec.89c05"],["c4c385d9.47b2a"],["8c188c08.aca008"]]},{"id":"432d3fc.4cce44","type":"function","z":"cfbd9ed9.13ff4","name":"整形","func":"var talk = '何だかわかりませんでした';\nmsg.payload = talk;\nreturn msg;","outputs":1,"noerr":0,"x":510,"y":840,"wires":[["839b33e7.cd55e"]]},{"id":"cd40e2ec.89c05","type":"function","z":"cfbd9ed9.13ff4","name":"整形","func":"var talk = '何だかわかりませんでした1';\nmsg.payload = talk;\nreturn msg;","outputs":1,"noerr":0,"x":510,"y":880,"wires":[["839b33e7.cd55e"]]},{"id":"c4c385d9.47b2a","type":"function","z":"cfbd9ed9.13ff4","name":"整形","func":"var talk = '私に見えているのは、'+ msg.payload.classes[0].class +'や'+ msg.payload.classes[1].class +'です';\nmsg.payload = talk;\nreturn msg;\n","outputs":1,"noerr":0,"x":510,"y":920,"wires":[["839b33e7.cd55e"]]},{"id":"8c188c08.aca008","type":"function","z":"cfbd9ed9.13ff4","name":"整形","func":"var talk = '私に見えているのは、'+ msg.payload.classes[0].class +'や'+ msg.payload.classes[1].class +'そして'+ msg.payload.classes[2].class +'などです';\nmsg.payload = talk;\nreturn msg;\n","outputs":1,"noerr":0,"x":510,"y":960,"wires":[["839b33e7.cd55e"]]},{"id":"d5187df1.e5eb98","type":"subflow:639f1e20.f58248","z":"cfbd9ed9.13ff4","name":"","x":870,"y":480,"wires":[]},{"id":"4b5b589d.ec85a","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(赤)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": true,\n \"g\": false,\n \"b\": false\n}","output":"json","x":540,"y":460,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"afa71489.e84ad","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(緑)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": false,\n \"g\": true,\n \"b\": false\n}","output":"json","x":540,"y":500,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"345c0019.0003e","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(青)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": false,\n \"g\": false,\n \"b\": true\n}","output":"json","x":540,"y":540,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"4625a0b1.a48868","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(水色)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": false,\n \"g\": true,\n \"b\": true\n}","output":"json","x":550,"y":620,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"1829d8a1.2f954f","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(黄)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": true,\n \"g\": true,\n \"b\": false\n}","output":"json","x":540,"y":580,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"2a13610b.c15116","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(マゼンタ)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": true,\n \"g\": false,\n \"b\": true\n}","output":"json","x":560,"y":660,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"426c4202.b3abd4","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(白)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": true,\n \"g\": true,\n \"b\": true\n}","output":"json","x":540,"y":700,"wires":[["d5187df1.e5eb98","bc436ccf.3691b8"]]},{"id":"8fe80c00.ff1168","type":"template","z":"cfbd9ed9.13ff4","name":"データの作成(オフ)","field":"payload","fieldType":"msg","format":"json","syntax":"plain","template":"{\n \"r\": false,\n \"g\": false,\n \"b\": false\n}","output":"json","x":1010,"y":700,"wires":[["d5187df1.e5eb98"]]},{"id":"4cc07b97.914de4","type":"switch","z":"cfbd9ed9.13ff4","name":"","property":"payload.entities[0].value","propertyType":"msg","rules":[{"t":"eq","v":"赤","vt":"str"},{"t":"eq","v":"緑","vt":"str"},{"t":"eq","v":"青","vt":"str"},{"t":"eq","v":"黄","vt":"str"},{"t":"eq","v":"水色","vt":"str"},{"t":"eq","v":"マゼンタ","vt":"str"},{"t":"eq","v":"白","vt":"str"}],"checkall":"true","repair":false,"outputs":7,"x":370,"y":580,"wires":[["4b5b589d.ec85a"],["afa71489.e84ad"],["345c0019.0003e"],["1829d8a1.2f954f"],["4625a0b1.a48868"],["2a13610b.c15116"],["426c4202.b3abd4"]]},{"id":"bc436ccf.3691b8","type":"delay","z":"cfbd9ed9.13ff4","name":"","pauseType":"delay","timeout":"2","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":840,"y":700,"wires":[["8fe80c00.ff1168"]]},{"id":"31725c4.46501a4","type":"trigger","z":"cfbd9ed9.13ff4","op1":"1","op2":"0","op1type":"str","op2type":"str","duration":"3","extend":false,"units":"s","reset":"","bytopic":"all","name":"","x":420,"y":80,"wires":[["97f9bbb3.2be49"]]},{"id":"88a417ca.f82b68","type":"switch","z":"cfbd9ed9.13ff4","name":"","property":"payload","propertyType":"msg","rules":[{"t":"eq","v":"1","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":250,"y":80,"wires":[["31725c4.46501a4","d58aee22.4c0f88"]]},{"id":"9b98b9a7.03ee68","type":"speakerpi-output","z":"cfbd9ed9.13ff4","choose":"streambased","filename":"","channels":"1","bitdepth":"16","samplerate":"22050","name":"","x":920,"y":1000,"wires":[[]]},{"id":"abf23073.1944b8","type":"debug","z":"cfbd9ed9.13ff4","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload.output.text[0]","x":720,"y":260,"wires":[]}]
読み込みに成功すると、下記のようなフローが作成されます。デプロイする前にいくつか準備することがありますので、デプロイは少々お待ちください。
Watson Assitantの準備とAPIキー
IBM CloudのダッシュボードにカタログからWatson Assistantを追加します。2018年の終盤に東京リージョンでもWatson Assistantが利用可能となりましたので、迷わず東京を選んでみましょう。
IBM Cloudのダッシュボードからカタログをクリックします。
AIカテゴリーからWatson Assitant(旧称Comversation)を選びます。
ロケーションで東京を選び、画面下の作成ボタンを押し、少し待ちます。
下記の画面に移ったら、ツールを起動します。下記の画面は後ほどAPI鍵のコピーに使いますので、閉じないでおていください。
ツールが起動すると、下記のような英語の画面になりますが、迷わずSkillsをクリックします。
下記の画面に移動したら、Create Newをクリックします。
ここからWatson AssistantのSkillのJSONファイルをダウンロードしておきます。
下記の画面が出たら、Import Skillをクリックし、Choose Json Fileを選び、先ほどダウンロードしたJSONファイルを選択して読み込みます。
読み込みが成功すると、下記のような画面が出て来ます。次に画面左上のSlillsをクリックします。
Skillsの画面で、縦にドットが3つ並んでいるところをクリックし、出て来たメニューのView API Detailsをクリックします。
Skill Datails画面に、Workspace IDが表示されますので、これをクリップボードにコピーします。
Node-Redに読み込んだ、先ほどのフローの中にあるAssistantノードをダブルクリックし、ノードの編集画面のWorkspace IDの欄に先ほどクリップボードに入れたWorkspace IDを貼り付けます。
Assitantのツールを起動させた画面を開き、API鍵とURLをそれぞれ前の画面のAPI KeyとSource Endpointにカット・アンド・ペーストします。Source Endpointが表示されていない場合は、Use Deault Service Endpointのチェックを外します。
同様に、Speech to Text、Text to SpeechそしてVisual RecognitionのAPI KeyとURLをセットします。（まだ海外のサイトを使用している場合は、これを機会に東京で作り直すことをお勧めします）
##フローをデプロイする
これでやっと今回のフローをデプロイすることができるようになりましたので、Node-Redのエディター画面の右上にあるデプロイボタンを押して、デプロイします。
##話しかけてみましょう
下記は、全体のフローの一部分ですが、PIN:↑22のノードでプッシュスイッチの状態を読み取っています。↑は、PIN22がプルアップされていて、通常時は"Hi"または、"1"のステータスになっていることを表しています。このノードは、PINの状態が変わると次のノードへ信号を伝達します。スイッチを押すと状態が、1から0に変化し、スイッチを話すと0から1に変化します。次にあるswitchノードでは、状態が1になった時に1のポートに出力を出すように設定していますので、スイッチを押した時は何もせず、離した時に次のノードへ信号を出します。そして、3秒間頭を緑色にして、3秒間録音をする命令が実行されます。
スイッチを押してから離し、頭が緑色の間に話しかけます。
たとえは、手を振ってとか、頭を赤にしてとか、あなたは誰？そして、冗談を言ってなどWatson AssistantのSkillに登録している内容に応じて色々な反応を返してくれます。
登録してある、Skillや今回のフローの解説は追って追記して行きたいと思いますので、ご要望がございましたらお知らせください。
###FacebookのTJBotFanページでコミュティーのみなさんと情報交換をしていますので、ご参加ください！