Azure CosmosDBとFIWARE Orionを連携させる

2020年6月に、Azure Managed ServiceとFIWAREを連携させるという記事を書きました。この時点ではFIWARE Orion 2.4.0で検証したのですが、Azure CosmosDBのMongoDB APIではOrionを動作させることができませんでした。

FIWARE OrionがMongoDBの細かい挙動に依存する実装になっていたため、多少のパッチ程度ではAzure CosmosDB上に実装されたMongoDB互換APIで動作させることができませんでした。

その後FIWARE Orionが3系にメジャーバージョンアップした際に、MongoDB 4系へ対応するためにDB連携周りのコードにもいくつか手が入りました。そこで今回は、最新のFIWARE Orion 3.4.0がAzure CosmosDBで動作するのか、再度検証したいと思います。

結果

先に結果を書きますと、全く問題無しというわけではないが、概ね動作することがわかりました。

Indexの問題

• Orionのデータベースのマニュアルにあるように、本来Orionはdatabaseやcollection、indexを自動生成します。しかしCosmosDBのMongoDB APIでは、その機能が動作しません。そのため、Indexを自力で作成しておく必要があります。
  • Orionが推奨するIndexについては、Orionのパフォーマンス・チューニングマニュアルを参照。
• 中の人曰く、Orionが自動生成する2つのIndexのうち、creDateというIndexがあればとりあえず動作は可能なようです。しかしCosmosDBの仕様上、 _id.servicePath, _id.id, _id.type というEntityIdとTypeで検索するためのネストされた複合Indexが設定できません。そのため自身のEntityの利用方法をもとに自力でイイカンジのIndexを設定しないと、パフォーマンス上のペナルティがあるかもしれません。

dateExpiresの問題

• Orionには有効期限を持ったEntityであるTransient Entityを作成する機能があります。これはMongoDBのTTL Index機能を用いて実現されていますが、CosmosDBのMongoDB APIには「TTL Indexは_tsフィールドのみ」という制約があります。そのためエンティティに有効期限を設定しても無視され、指定した期限が来てもエンティティは自動削除されません。

time=2022-01-03T13:16:39.180Z | lvl=ERROR | corr=62900700-6c97-11ec-a8c3-0242ac130002 | trans=1641171277-902-00000000126 | from=172.19.0.1 | srv=tenant_a | subsrv=<none> | comp=Orion | op=AlarmManager.cpp[185]:dbError | msg=Raising alarm DatabaseError: collection: oriondb-tenant_a.entities - runCommand(): { "createIndexes" : "entities", "indexes" : [ { "key" : { "expDate" : 1 }, "name" : "expDate_1", "expireAfterSeconds" : 0 } ] } - exception: The 'expireAfterSeconds' option is supported on '_ts' field only.

検証環境のインフラ構成

インフラ構成は次図のようにしました。CosmosDBをインターネットに公開し、そこへインターネット経由で手元のOrionから接続することも可能ではありますが、データベースがむき出しになるのは好みではありません。そこで今回は、Azure VM上にOrionを立て、インターネットに公開されていないCosmosDBへサービスエンドポイント経由で接続するようにしました。
また検証用にOrion以外のプロセスも必要となるため、dockerコンテナとして動作させることにしました。

検証環境のバージョン

	バージョン
Azure CLI	2.31.0
CosmosDBのMongoDB API	4.0
FIWARE Orion	3.4.0

FIWAREのパラメータ

パラメータ	値
database	oriondb
fiware-service	tenant_a
fiware-servicepath	指定無し

OrionはEntityをMongoDBに格納する際、以下のようなルールでdatabaseを決めます（MULTISERVICE/MULTITENANT DATABASE SEPARATION）。

• デフォルトテナントのEntityは<db>というdatabaseに格納
• FIWARE-SERVICEによってテナントが指定されたEntityは<db>-<tenant>というdatabaseに格納

今回はDB名をoriondb、テナント名をtenant_aにすることを前提に、命名規則に従ってdatabaseや指定したIndexを持つcollectionをazコマンドから生成しておきます。

検証手順

環境構築

リソースグループの準備

リソースグループの作成

$ az group create \
  --name orionCosmosDB \
  --location japaneast

VNETの準備

VNETの作成

$ az network vnet create \
  --resource-group orionCosmosDB \
  --name orionCosmosDBVNet \
  --address-prefix 10.0.0.0/16

CosmosDBへのサービスエンドポイントを持ったサブネットの作成

$ az network vnet subnet create \
  --resource-group orionCosmosDB \
  --vnet-name orionCosmosDBVNet \
  --name orionCosmosDBSubnet \
  --address-prefix 10.0.1.0/24 \
  --service-endpoints Microsoft.AzureCosmosDB

SSHのみ許可したセキュリティグループの作成

$ az network nsg create \
  --resource-group orionCosmosDB \
  --name orionCosmosDBNSG

$ az network nsg rule create \
  --resource-group orionCosmosDB \
  --nsg-name orionCosmosDBNSG \
  --name ssh \
  --access allow \
  --protocol Tcp \
  --direction Inbound \
  --priority 200 \
  --source-address-prefix "*" \
  --source-port-range "*" \
  --destination-address-prefix "*" \
  --destination-port-range 22

サービスエンドポイントのIDを確認

$ AzureCosmosDBEndpoint=$(az network vnet subnet show \
  --resource-group orionCosmosDB \
  --vnet-name orionCosmosDBVNet \
  --name orionCosmosDBSubnet \
  --query 'id' \
  --output tsv); \
  echo ${AzureCosmosDBEndpoint}

MongoDB APIを持つCosmosDBを準備

CosmosDBの作成

$ az cosmosdb create \
  --resource-group orionCosmosDB \
  --name orioncosmosdbaccount \
  --kind MongoDB \
  --server-version 4.0 \
  --default-consistency-level Eventual \
  --locations regionName=japaneast failoverPriority=0 isZoneRedundant=False \
  --capabilities EnableServerless \
  --enable-virtual-network true \
  --virtual-network-rules ${AzureCosmosDBEndpoint}

databaseの作成

$ az cosmosdb mongodb database create \
  --resource-group orionCosmosDB \
  --account-name orioncosmosdbaccount \
  --name oriondb-tenant_a

collectionとindexの作成

$ cat << __EOF__ > oriondb-tenant_a-idx.json
[
    {
        "key": {"keys": ["creDate"]}
    }
]
__EOF__

$ az cosmosdb mongodb collection create \
--resource-group orionCosmosDB \
--account-name orioncosmosdbaccount \
--database-name oriondb-tenant_a \
--name entities \
--idx @oriondb-tenant_a-idx.json

$ rm oriondb-tenant_a-idx.json

接続文字列の取得

$ AzureConnectionString=$(az cosmosdb keys list \
  --resource-group orionCosmosDB \
  --name orioncosmosdbaccount \
  --type connection-strings \
  --query "connectionStrings[?description == 'Primary MongoDB Connection String']" \
  | jq -r '.[].connectionString'); \
  echo ${AzureConnectionString}

Orion用のVMの準備

VMの作成

ssh-keygenで鍵ペアは生成済みだとします。

$ az vm create \
  --resource-group orionCosmosDB \
  --name clientVM \
  --vnet-name orionCosmosDBVNet \
  --subnet orionCosmosDBSubnet \
  --nsg orionCosmosDBNSG \
  --public-ip-sku Standard \
  --image UbuntuLTS \
  --size Standard_A2_v2 \
  --admin-username ubuntu \
  --ssh-key-values ~/.ssh/azure.pub

パブリックIPの確認

$ AzureVMIP=$(az vm list-ip-addresses \
  --resource-group orionCosmosDB \
  --query "[].virtualMachine.network.publicIpAddresses[?name == 'clientVMPublicIP']" \
  | jq -r '.[][].ipAddress'); \
  echo ${AzureVMIP}

CosmosDBへの接続文字列を渡してVMへSSH

az cosmosdb keys listで取得したCosmosDBへの接続文字列を環境変数として引き渡し、bashをVM上に起動します。

$ ssh -i ~/.ssh/azure -t ubuntu@${AzureVMIP} \
  "AzureConnectionString=$(printf %q "$AzureConnectionString") bash"

以下はVM上での作業です。

dockerとdocker-composeをインストール

ubuntu@clientVM:~$ sudo apt update && sudo apt upgrade -y

ubuntu@clientVM:~$ sudo apt install ca-certificates curl gnupg lsb-release -y

ubuntu@clientVM:~$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
                   | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

ubuntu@clientVM:~$ echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

ubuntu@clientVM:~$ sudo apt update && sudo apt-get install docker-ce docker-ce-cli containerd.io -y

ubuntu@clientVM:~$ sudo docker run hello-world

ubuntu@clientVM:~$ sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" \
  -o /usr/local/bin/docker-compose

ubuntu@clientVM:~$ sudo chmod +x /usr/local/bin/docker-compose

ubuntu@clientVM:~$ sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

ubuntu@clientVM:~$ docker-compose --version

mongo shellの起動スクリプトを生成

ubuntu@clientVM:~$ cat << __EOF__ > mongo_shell.bash
# !/bin/bash

sudo docker run -it mongo:4.4 mongo "${AzureConnectionString}"
__EOF__

ubuntu@clientVM:~$ chmod +x mongo_shell.bash

通知先サーバのDockerfileを生成

ubuntu@clientVM:~$ cat << '__EOF__' > Dockerfile
FROM python:2.7-slim
LABEL MAINTAINER Nobuyuki Matsui <[e-mail]>

RUN apt update && apt upgrade -y && \
    apt install wget -y && \
    wget https://raw.githubusercontent.com/telefonicaid/fiware-orion/master/scripts/accumulator-server.py -O /opt/accumulator-server.py && \
    chmod +x /opt/accumulator-server.py && \
    pip install Flask==1.0.2 pyOpenSSL==19.0.0 paho-mqtt==1.5.1

ENV PYTHONUNBUFFERED 1

WORKDIR /opt
ENTRYPOINT ["/opt/accumulator-server.py"]
CMD ["--port", "1028", "--url", "/accumulate", "--host", "0.0.0.0", "--pretty-print", "-v"]
__EOF__

orionと通知先サーバを起動するdocker-compose.ymlを生成

ubuntu@clientVM:~$ USER=$(echo $AzureConnectionString \
                   | sed -r 's/^mongodb:\/\/([^:]+):([^@]+)@([^/]+).*$/\1/')

ubuntu@clientVM:~$ PASSWORD=$(echo $AzureConnectionString \
                   | sed -r 's/^mongodb:\/\/([^:]+):([^@]+)@([^/]+).*$/\2/')

ubuntu@clientVM:~$ HOST=$(echo $AzureConnectionString \
                   | sed -r 's/^mongodb:\/\/([^:]+):([^@]+)@([^/]+).*$/\3/')

ubuntu@clientVM:~$ cat << __EOF__ > docker-compose.yml
version: "3"

services:
  orion:
    image: fiware/orion:3.4.0
    ports:
      - "1026:1026"
    command: -logLevel "debug" -db "oriondb" -dbSSL -dbDisableRetryWrites -dbhost "${HOST}" -rplSet "globaldb" -dbuser "${USER}" -dbpwd "${PASSWORD}"

  accumulator:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "1028:1028"
__EOF__

VM上のmongo shellからの確認

mongo container経由でCosmosDBのMongoDB APIに接続

ubuntu@clientVM:~$ ./mongo_shell.bash

正しく環境構築できていれば、globaldb:PRIMARY>とプロンプトが表示されます。

MongoDB shell version v4.4.11
connecting to: mongodb://orioncosmosdbaccount.mongo.cosmos.azure.com:10255/?appName=%40orioncosmosdbaccount%40&compressors=disabled&gssapiServiceName=mongodb&maxIdleTimeMS=120000&replicaSet=globaldb&retrywrites=false&ssl=true
Implicit session: session { "id" : UUID("628ae6c3-d4f6-42d6-9617-d70e058c5f96") }
MongoDB server version: 4.0.0
WARNING: shell and server versions do not match
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
	https://docs.mongodb.com/
Questions? Try the MongoDB Developer Community Forums
	https://community.mongodb.com
globaldb:PRIMARY>

database、collection、indexの確認

globaldb:PRIMARY> show dbs

oriondb-tenant_a  0.000GB

globaldb:PRIMARY> use oriondb-tenant_a

switched to db oriondb-tenant_a

globaldb:PRIMARY> show collections

entities

oriondb-tenant_adatabaseにentitiescollectionが存在することが確認できます。

globaldb:PRIMARY> db.entities.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "oriondb-tenant_a.entities"
	},
	{
		"v" : 1,
		"key" : {
			"creDate" : 1
		},
		"name" : "creDate_1",
		"ns" : "oriondb-tenant_a.entities"
	}
]

entitiescollectionにcreDate`インデックスが存在していることが確認できます。

globaldb:PRIMARY> db.entities.find({})

entitiescollection`にはまだドキュメントが存在しません。

globaldb:PRIMARY> exit

mongo shellから抜けます。

VM上にorionを起動

docker-composeでorionを起動

ubuntu@clientVM:~$ sudo docker-compose up -d

orionが無事に起動すると、msg=Startup completedというメッセージが表示されるはずです。

ubuntu@clientVM:~$ sudo docker logs ubuntu_orion_1

...
time=2022-01-02T23:58:32.892Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1088]:main | msg=start command line </usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -disableFileLog -logLevel debug -db oriondb -dbSSL -dbDisableRetryWrites -dbhost orioncosmosdbaccount.mongo.cosmos.azure.com:10255 -rplSet globaldb -dbuser orioncosmosdbaccount -dbpwd ******>
time=2022-01-02T23:58:32.892Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1162]:main | msg=Orion Context Broker is running
time=2022-01-02T23:58:36.153Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=mongoConnectionPool.cpp[506]:mongoConnectionPoolInit | msg=Connected to mongodb://orioncosmosdbaccount:******@orioncosmosdbaccount.mongo.cosmos.azure.com:10255/?replicaSet=globaldb&tls=true&tlsAllowInvalidCertificates=true&retryWrites=false&connectTimeoutMS=10000 (dbName: oriondb, poolsize: 10)
time=2022-01-02T23:58:36.349Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=AlarmManager.cpp[185]:dbError | msg=Raising alarm DatabaseError: collection: oriondb-tenant_a.entities - runCommand(): { "createIndexes" : "entities", "indexes" : [ { "key" : { "expDate" : 1 }, "name" : "expDate_1", "expireAfterSeconds" : 0 } ] } - exception: The 'expireAfterSeconds' option is supported on '_ts' field only.
time=2022-01-02T23:58:36.354Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=AlarmManager.cpp[209]:dbErrorReset | msg=Releasing alarm DatabaseError
time=2022-01-02T23:58:36.438Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1294]:main | msg=Startup completed

ただし上記のように、Indexの生成でエラーが出ているかもしれません。これはCosmosDBのIndex仕様によるものです。あきらめてください。

OrionのNGSIv2 APIを検証

それではFIWARE NGSI APIV2 WALKTHROUGHに沿って、OrionのAPIが一通り動作していることを確認しましょう。

コンテキスト管理

エンティティの作成

ubuntu@clientVM:~$ curl localhost:1026/v2/entities -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "id": "Room1",
  "type": "Room",
  "temperature": {
    "value": 23,
    "type": "Float"
  },
  "pressure": {
    "value": 720,
    "type": "Integer"
  }
}
EOF

ubuntu@clientVM:~$ curl localhost:1026/v2/entities -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "id": "Room2",
  "type": "Room",
  "temperature": {
    "value": 21,
    "type": "Float"
  },
  "pressure": {
    "value": 711,
    "type": "Integer"
  }
}
EOF

リクエストに成功し、HTTP/1.1 201 Createdが返ってくるはずです。
またmongo shellからCosmosDBにクエリを投げると、上記２つのドキュメントが格納されていることが確認できます。

ubuntu@clientVM:~$ ./mongo_shell.bash

globaldb:PRIMARY> use oriondb-tenant_a

globaldb:PRIMARY> db.entities.find({})

{ "_id" : { "id" : "Room1", "type" : "Room", "servicePath" : "/" }, "attrNames" : [ "temperature", "pressure" ], "attrs" : { "temperature" : { "type" : "Float", "creDate" : 1641171853.6182125, "modDate" : 1641171853.6182125, "value" : 23, "mdNames" : [ ] }, "pressure" : { "type" : "Integer", "creDate" : 1641171853.6182125, "modDate" : 1641171853.6182125, "value" : 720, "mdNames" : [ ] } }, "creDate" : 1641171853.6182125, "modDate" : 1641171853.6182125, "lastCorrelator" : "10faf7d0-6c31-11ec-ad66-0242ac130002" }
{ "_id" : { "id" : "Room2", "type" : "Room", "servicePath" : "/" }, "attrNames" : [ "temperature", "pressure" ], "attrs" : { "temperature" : { "type" : "Float", "creDate" : 1641171975.6809523, "modDate" : 1641171975.6809523, "value" : 21, "mdNames" : [ ] }, "pressure" : { "type" : "Integer", "creDate" : 1641171975.6809523, "modDate" : 1641171975.6809523, "value" : 711, "mdNames" : [ ] } }, "creDate" : 1641171975.6809523, "modDate" : 1641171975.6809523, "lastCorrelator" : "59c165bc-6c31-11ec-a31a-0242ac130002" }

エンティティの取得

Room1エンティティ全体を取得

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1?type=Room -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "id": "Room1",
    "pressure": {
        "metadata": {},
        "type": "Integer",
        "value": 720
    },
    "temperature": {
        "metadata": {},
        "type": "Float",
        "value": 23
    },
    "type": "Room"
}

Room1エンティティの属性値のみ取得

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "id": "Room1",
    "pressure": 720,
    "temperature": 23,
    "type": "Room"
}

ubuntu@clientVM:~$ curl 'localhost:1026/v2/entities/Room1?options=values&attrs=temperature,pressure' -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    23,
    720
]

特定の属性のみ取得

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/temperature -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "metadata": {},
    "type": "Float",
    "value": 23
}

特定の属性の値のみ取得

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/temperature/value -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: text/plain' \
                   | python -mjson.tool

23

存在しないエンティティ

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room5 -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

...
< HTTP/1.1 404 Not Found
...
{
    "description": "The requested entity has not been found. Check type and id",
    "error": "NotFound"
}

存在しない属性

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/humidity -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

...
< HTTP/1.1 404 Not Found
...
{
    "description": "The entity does not have such an attribute",
    "error": "NotFound"
}

エンティティのフィルタリング

すべてのエンティティの取得

ubuntu@clientVM:~$ curl localhost:1026/v2/entities -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room1",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 720
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 23
        },
        "type": "Room"
    },
    {
        "id": "Room2",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 711
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 21
        },
        "type": "Room"
    }
]

ubuntu@clientVM:~$ curl localhost:1026/v2/entities?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room1",
        "pressure": 720,
        "temperature": 23,
        "type": "Room"
    },
    {
        "id": "Room2",
        "pressure": 711,
        "temperature": 21,
        "type": "Room"
    }
]

typeでフィルタリング

ubuntu@clientVM:~$ curl localhost:1026/v2/entities?type=Room -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room1",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 720
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 23
        },
        "type": "Room"
    },
    {
        "id": "Room2",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 711
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 21
        },
        "type": "Room"
    }
]

IDパターンでフィルタリング

ubuntu@clientVM:~$ curl localhost:1026/v2/entities?idPattern=^Room[2-5] -g -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room2",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 711
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 21
        },
        "type": "Room"
    }
]

属性の条件でフィルタリング

ubuntu@clientVM:~$ curl 'localhost:1026/v2/entities?q=temperature>22' -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room1",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 720
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 23
        },
        "type": "Room"
    }
]

エンティティの更新

属性の更新

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -X PATCH -d @- <<EOF
{
  "temperature": {
    "value": 26.5,
    "type": "Float"
  },
  "pressure": {
    "value": 763,
    "type": "Float"
  }
}
EOF

< HTTP/1.1 204 No Content

属性値の更新

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/temperature/value -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: text/plain' \
                   -X PUT -d 28.5

< HTTP/1.1 204 No Content

### サブスクリプション

サブスクリプションの登録と一覧取得

Room1のpressureが変更されると、temperatureを通知するsubscriptionを登録

ubuntu@clientVM:~$ curl localhost:1026/v2/subscriptions -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "description": "A subscription to get info about Room1",
  "subject": {
    "entities": [
      {
        "id": "Room1",
        "type": "Room"
      }
    ],
    "condition": {
      "attrs": [
        "pressure"
      ]
    }
  },
  "notification": {
    "http": {
      "url": "http://accumulator:1028/accumulate"
    },
    "attrs": [
      "temperature"
    ]
  },
  "expires": "2040-01-01T14:00:00.00Z"
}
EOF

< HTTP/1.1 201 Created
< Connection: Keep-Alive
< Content-Length: 0
< Location: /v2/subscriptions/61d2636b5a72ee5ace05f95c
< Fiware-Correlator: 4ab4ed24-6c3f-11ec-ab7e-0242ac130002
< Date: Mon, 03 Jan 2022 02:46:03 GMT

subscriptionの一覧を取得

ubuntu@clientVM:~$ curl localhost:1026/v2/subscriptions -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "description": "A subscription to get info about Room1",
        "expires": "2040-01-01T14:00:00.000Z",
        "id": "61d2636b5a72ee5ace05f95c",
        "notification": {
            "attrs": [
                "temperature"
            ],
            "attrsFormat": "normalized",
            "http": {
                "url": "http://accumulator:1028/accumulate"
            },
            "onlyChangedAttrs": false
        },
        "status": "active",
        "subject": {
            "condition": {
                "attrs": [
                    "pressure"
                ]
            },
            "entities": [
                {
                    "id": "Room1",
                    "type": "Room"
                }
            ]
        }
    }
]

通知されるようにエンティティを更新

Room1のpressureを更新

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/pressure/value -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: text/plain' \
                   -X PUT -d 802

< HTTP/1.1 204 No Content

通知サーバのログを確認すると、登録したsubscriptionから通知が送信されたことが確認できます。

通知内容の確認

ubuntu@clientVM:~$ sudo docker logs ubuntu_accumulator_1

...
POST http://accumulator:1028/accumulate
Fiware-Servicepath: /
Content-Length: 141
User-Agent: orion/3.4.0 libcurl/7.61.1
Ngsiv2-Attrsformat: normalized
Host: accumulator:1028
Accept: application/json
Fiware-Service: tenant_a
Content-Type: application/json; charset=utf-8
Fiware-Correlator: a043c616-6c3f-11ec-a6b0-0242ac130002; cbnotif=1

{
    "data": [
        {
            "id": "Room1",
            "temperature": {
                "metadata": {},
                "type": "Float",
                "value": 28.5
            },
            "type": "Room"
        }
    ],
    "subscriptionId": "61d2636b5a72ee5ace05f95c"
}
=======================================

172.19.0.2 - - [03/Jan/2022 02:48:26] "POST /accumulate HTTP/1.1" 200 -

フィルタリング式付きサブスクリプションの検証

Room1のpressureが700~800の範囲内で変更されると、temperatureを通知するsubscriptionを登録

ubuntu@clientVM:~$ curl localhost:1026/v2/subscriptions -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "description": "A subscription to get info about Room1 with filtering expresion",
  "subject": {
    "entities": [
      {
        "id": "Room1",
        "type": "Room"
      }
    ],
    "condition": {
      "attrs": [
        "pressure"
      ],
      "expression": {
        "q": "pressure:700..800"
      }
    }
  },
  "notification": {
    "http": {
      "url": "http://accumulator:1028/accumulate"
    },
    "attrs": [
      "temperature"
    ]
  },
  "expires": "2040-01-01T14:00:00.00Z"
}
EOF

< HTTP/1.1 201 Created
< Connection: Keep-Alive
< Content-Length: 0
< Location: /v2/subscriptions/61d2ac6b5a72ee5ace05f95f
< Fiware-Correlator: cde16c4c-6c6a-11ec-b4fa-0242ac130002
< Date: Mon, 03 Jan 2022 07:57:31 GMT

次の更新はフィルタ式にヒットするため、61d2636b5a72ee5ace05f95cと61d2ac6b5a72ee5ace05f95fは共に通知を送信

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/pressure/value -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: text/plain' \
                   -X PUT -d 715

POST http://accumulator:1028/accumulate
Fiware-Servicepath: /
Content-Length: 141
User-Agent: orion/3.4.0 libcurl/7.61.1
Ngsiv2-Attrsformat: normalized
Host: accumulator:1028
Accept: application/json
Fiware-Service: tenant_a
Content-Type: application/json; charset=utf-8
Fiware-Correlator: f5396e5c-6c6a-11ec-8f10-0242ac130002; cbnotif=1

{
    "data": [
        {
            "id": "Room1",
            "temperature": {
                "metadata": {},
                "type": "Float",
                "value": 28.5
            },
            "type": "Room"
        }
    ],
    "subscriptionId": "61d2636b5a72ee5ace05f95c"
}
=======================================

172.19.0.2 - - [03/Jan/2022 07:58:37] "POST /accumulate HTTP/1.1" 200 -
POST http://accumulator:1028/accumulate
Fiware-Servicepath: /
Content-Length: 141
User-Agent: orion/3.4.0 libcurl/7.61.1
Ngsiv2-Attrsformat: normalized
Host: accumulator:1028
Accept: application/json
Fiware-Service: tenant_a
Content-Type: application/json; charset=utf-8
Fiware-Correlator: f5396e5c-6c6a-11ec-8f10-0242ac130002; cbnotif=2

{
    "data": [
        {
            "id": "Room1",
            "temperature": {
                "metadata": {},
                "type": "Float",
                "value": 28.5
            },
            "type": "Room"
        }
    ],
    "subscriptionId": "61d2ac6b5a72ee5ace05f95f"
}
=======================================

172.19.0.2 - - [03/Jan/2022 07:58:37] "POST /accumulate HTTP/1.1" 200 -

次の更新はフィルタ式にヒットしないため、61d2636b5a72ee5ace05f95cのみ通知を送信

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room1/attrs/pressure/value -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: text/plain' \
                   -X PUT -d 801

POST http://accumulator:1028/accumulate
Fiware-Servicepath: /
Content-Length: 141
User-Agent: orion/3.4.0 libcurl/7.61.1
Ngsiv2-Attrsformat: normalized
Host: accumulator:1028
Accept: application/json
Fiware-Service: tenant_a
Content-Type: application/json; charset=utf-8
Fiware-Correlator: 25b1fed2-6c6b-11ec-ba6b-0242ac130002; cbnotif=1

{
    "data": [
        {
            "id": "Room1",
            "temperature": {
                "metadata": {},
                "type": "Float",
                "value": 28.5
            },
            "type": "Room"
        }
    ],
    "subscriptionId": "61d2636b5a72ee5ace05f95c"
}
=======================================

172.19.0.2 - - [03/Jan/2022 07:59:59] "POST /accumulate HTTP/1.1" 200 -

型情報の取得

すべての型の情報を取得

`type=Room`のエンティティしか登録していないので、１種類のみ

ubuntu@clientVM:~$ curl localhost:1026/v2/types -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "attrs": {
            "pressure": {
                "types": [
                    "Float",
                    "Integer"
                ]
            },
            "temperature": {
                "types": [
                    "Float"
                ]
            }
        },
        "count": 2,
        "type": "Room"
    }
]

Room型の情報を取得

ubuntu@clientVM:~$ curl localhost:1026/v2/types/Room -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "attrs": {
        "pressure": {
            "types": [
                "Float",
                "Integer"
            ]
        },
        "temperature": {
            "types": [
                "Float"
            ]
        }
    },
    "count": 2
}

バッチ処理

バッチで追加

ubuntu@clientVM:~$ curl localhost:1026/v2/op/update -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "actionType": "append",
  "entities": [
    {
      "type": "Room",
      "id": "Room3",
      "temperature": {
        "value": 21.2,
        "type": "Float"
      },
      "pressure": {
        "value": 722,
        "type": "Integer"
      }
    },
    {
      "type": "Room",
      "id": "Room4",
      "temperature": {
        "value": 31.8,
        "type": "Float"
      },
      "pressure": {
        "value": 712,
        "type": "Integer"
      }
    }
  ]
}
EOF

< HTTP/1.1 204 No Content

Room3とRoom4を一挙に追加

ubuntu@clientVM:~$ curl localhost:1026/v2/entities?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room1",
        "pressure": 801,
        "temperature": 28.5,
        "type": "Room"
    },
    {
        "id": "Room2",
        "pressure": 711,
        "temperature": 21,
        "type": "Room"
    },
    {
        "id": "Room3",
        "pressure": 722,
        "temperature": 21.2,
        "type": "Room"
    },
    {
        "id": "Room4",
        "pressure": 712,
        "temperature": 31.8,
        "type": "Room"
    }
]

バッチで更新

ubuntu@clientVM:~$ curl localhost:1026/v2/op/update -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "actionType": "update",
  "entities": [
    {
      "type": "Room",
      "id": "Room3",
      "temperature": {
        "value": 29.9,
        "type": "Float"
      }
    },
    {
      "type": "Room",
      "id": "Room4",
      "pressure": {
        "value": 709,
        "type": "Integer"
      }
    }
  ]
}
EOF

< HTTP/1.1 204 No Content

Room3とRoom4を一挙に更新

ubuntu@clientVM:~$ curl localhost:1026/v2/entities?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "id": "Room1",
        "pressure": 801,
        "temperature": 28.5,
        "type": "Room"
    },
    {
        "id": "Room2",
        "pressure": 711,
        "temperature": 21,
        "type": "Room"
    },
    {
        "id": "Room3",
        "pressure": 722,
        "temperature": 29.9,
        "type": "Room"
    },
    {
        "id": "Room4",
        "pressure": 709,
        "temperature": 31.8,
        "type": "Room"
    }
]

バッチで検索

ubuntu@clientVM:~$ cat << EOF | curl localhost:1026/v2/op/query -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- \
                   | python -mjson.tool
{
  "entities": [
    {
      "idPattern": "Room[1-2]",
      "type": "Room"
    }
  ],
  "attrs": [
    "temperature",
    "pressure"
  ],
  "expression": {
    "q": "pressure==700..720"
  }
}
EOF

[
    {
        "id": "Room2",
        "pressure": {
            "metadata": {},
            "type": "Integer",
            "value": 711
        },
        "temperature": {
            "metadata": {},
            "type": "Float",
            "value": 21
        },
        "type": "Room"
    }
]

レジストレーション

docker-composeで別のOrion（provider-orion）を起動し、Room5エンティティを登録します。

provider-orion用のdocker-compose.ymlを生成

ubuntu@clientVM:~$ cat << __EOF__ > provider-docker-compose.yml
version: "3"

services:
  provider-orion:
    image: fiware/orion:3.4.0
    ports:
      - "1027:1026"
    depends_on:
      - mongo
    command: -dbhost mongo
    networks:
      - ubuntu_default

  mongo:
    image: mongo:4.4
    command: --nojournal
    networks:
      - ubuntu_default

networks:
  ubuntu_default:
    external: true
__EOF__

provider-orionを起動

ubuntu@clientVM:~$ sudo docker-compose -f provider-docker-compose.yml up -d

provider-orionにRoom5を作成

provider-orionにRoom5を作成し、登録されていることを確認

ubuntu@clientVM:~$ curl localhost:1027/v2/entities -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @- <<EOF
{
  "id": "Room5",
  "type": "Room",
  "temperature": {
    "value": 12.3,
    "type": "Float"
  },
  "pressure": {
    "value": 718,
    "type": "Integer"
  }
}
EOF

ubuntu@clientVM:~$ curl localhost:1027/v2/entities/Room5?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "id": "Room5",
    "pressure": 718,
    "temperature": 12.3,
    "type": "Room"
}

CosmosDBをバックエンドとする元のOrionにRoom5は存在しない

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room5?options=keyValues -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

< HTTP/1.1 404 Not Found
...
{
    "description": "The requested entity has not been found. Check type and id",
    "error": "NotFound"
}

Room5のレジストレーションを登録し、元のOrion経由でRoom5の情報が取得できることを確認します。

レジストレーションを作成

ubuntu@clientVM:~$ curl localhost:1026/v2/registrations -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -d @-  <<EOF
{
  "description": "Registration for Room5",
  "dataProvided": {
    "entities": [
      {
        "id": "Room5",
        "type": "Room"
      }
    ],
    "attrs": [
      "temperature",
      "pressure"
    ]
  },
  "provider": {
    "http": {
      "url": "http://provider-orion:1026/v2"
    }
  }
}
EOF

< HTTP/1.1 201 Created
< Connection: Keep-Alive
< Content-Length: 0
< Location: /v2/registrations/61d2eeaa5a72ee5ace05f961
< Fiware-Correlator: 49e9cbb4-6c92-11ec-bde4-0242ac130002
< Date: Mon, 03 Jan 2022 12:40:10 GMT

レジストレーションを検索

ubuntu@clientVM:~$ curl localhost:1026/v2/registrations -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

[
    {
        "dataProvided": {
            "attrs": [
                "temperature",
                "pressure"
            ],
            "entities": [
                {
                    "id": "Room5",
                    "type": "Room"
                }
            ]
        },
        "description": "Registration for Room5",
        "id": "61d2eeaa5a72ee5ace05f961",
        "provider": {
            "http": {
                "url": "http://provider-orion:1026/v2"
            },
            "legacyForwarding": false,
            "supportedForwardingMode": "all"
        },
        "status": "active"
    }
]

レジストレーションを通じてRoom5の情報を取得

先程は404になっていたRoom5の情報を、レジストレーション経由で取得

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room5?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "id": "Room5",
    "pressure": 718,
    "temperature": 12.3,
    "type": "Room"
}

レジストレーションを通じてRoom5の情報を更新

ubuntu@clientVM:~$ curl localhost:1026/v2/entities/Room5/attrs?type=Room -v \
                   -H 'fiware-service: tenant_a' \
                   -H 'Content-Type: application/json' \
                   -X PATCH -d @- <<EOF
{
  "temperature": {
    "value": 10.8,
    "type": "Float"
  },
  "pressure": {
    "value": 721,
    "type": "Float"
  }
}
EOF

< HTTP/1.1 204 No Content

provider-orionのRoom5が更新されている

ubuntu@clientVM:~$ curl localhost:1027/v2/entities/Room5?options=keyValues -s -S \
                   -H 'fiware-service: tenant_a' \
                   -H 'Accept: application/json' \
                   | python -mjson.tool

{
    "id": "Room5",
    "pressure": 721,
    "temperature": 10.8,
    "type": "Room"
}

まとめ

ということで、CosmosDBに上手くIndexを設定すれば、全く問題無しというわけではないが、NGSIv2のAPIは概ね動作することがわかりました。
ただし今回は、geolocationalなフィルタリングやメタデータの検索など、NGSIv2が備える全ての機能を検証できたわけではありません。またパフォーマンスも評価できていないですし、プロダクションで利用する場合にはさらなる検証を重ねる必要がありそうです。