DynamoDB に boto3 で接続 （ハッシュキー、レンジキーを使う）

次のページと同様ですが、ハッシュキーとレンジキーを使うサンプルです。
ハッシュ + レンジ　がプライマリーキーになります。

ローカルのDynamoDB に、python3 で CRUD を行う
参考にしたページ
Interfacing Amazon DynamoDB with Python using Boto3

このサンプルでは、ローカルの DynamoDB にアクセスします。

テーブルの作成
tbl_iot というテーブルを作成します。
ハッシュキー　は id_device
レンジキーは、 timestamp
です。

create_table.sh

#
aws dynamodb create-table --table-name 'tbl_iot' \
    --attribute-definitions \
        AttributeName=id_device,AttributeType=S \
        AttributeName=timestamp,AttributeType=S \
    --key-schema \
        AttributeName=id_device,KeyType=HASH \
        AttributeName=timestamp,KeyType=RANGE \
    --provisioned-throughput \
        ReadCapacityUnits=5,WriteCapacityUnits=5 \
    --endpoint-url http://localhost:8000
#

やり直す時のテーブルの削除

delete_table.sh

aws dynamodb delete-table --table-name tbl_iot --endpoint-url http://localhost:8000

テーブルの一覧を表示

list_table.sh

aws dynamodb list-tables --endpoint-url http://localhost:8000

データの挿入

insert_data.sh

#
#aws dynamodb list-tables --endpoint-url http://localhost:8000
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0001" }, "timestamp": { "S": "20180122082400" }, "temperature": { "N": "21.15" }, "humidity": { "N": "51.62" } }' \
    --endpoint-url http://localhost:8000
#
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0002" }, "timestamp": { "S": "20180122082100" }, "temperature": { "N": "22.25" }, "humidity": { "N": "50.51" } }' \
    --endpoint-url http://localhost:8000
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0002" }, "timestamp": { "S": "20180122082200" }, "temperature": { "N": "23.45" }, "humidity": { "N": "49.82" } }' \
    --endpoint-url http://localhost:8000
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0002" }, "timestamp": { "S": "20180122082300" }, "temperature": { "N": "24.55" }, "humidity": { "N": "53.15" } }' \
    --endpoint-url http://localhost:8000
#
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0003" }, "timestamp": { "S": "20180122082100" }, "temperature": { "N": "25.43" }, "humidity": { "N": "60.21" } }' \
    --endpoint-url http://localhost:8000
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0003" }, "timestamp": { "S": "20180122082200" }, "temperature": { "N": "27.25" }, "humidity": { "N": "59.42" } }' \
    --endpoint-url http://localhost:8000
#
aws dynamodb put-item --table-name tbl_iot --item \
    '{ "id_device": { "S": "ras0003" }, "timestamp": { "S": "20180122082300" }, "temperature": { "N": "26.15" }, "humidity": { "N": "61.26" } }' \
    --endpoint-url http://localhost:8000
#

ここからが、boto3 を使った python3 のプログラムです。

全てのデータを表示

scan01.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   scan01.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3

# --------------------------------------------------------------------
from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

response = table.scan()
display_proc(response['Items'])
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

display.py

# -*- coding: utf-8 -*-
#
#   display.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys

# --------------------------------------------------------------------
def display_proc(items):
    for it in items:
        str_out = it['id_device'] + '\t'
        str_out += it['timestamp']  + '\t'
        str_out += str(it['temperature']) + '\t'
        str_out += str(it['humidity'])
        print(str_out)
#
# --------------------------------------------------------------------

実行結果

特定のハッシュキーを持つものを表示
query を使用
この例では、ras0002 というキーのものを表示します。
同じことを scan を使ってもできます。(scan02.py)

query01.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   query01.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from boto3.dynamodb.conditions import Key

from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

id_in="ras0002"
sys.stderr.write ("id_in = " + id_in + "\n")
response = table.query(KeyConditionExpression=Key('id_device').eq(id_in))

display_proc(response['Items'])

sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

実行結果

ハッシュキーと、レンジキーでひとつのデータを表示
get_item を使用

get_item01.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   get_item01.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from boto3.dynamodb.conditions import Key

from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

id_in="ras0002"
timestamp_in="20180122082300"
response = table.get_item(Key={'id_device': id_in,'timestamp': timestamp_in})
#
items=[response['Item']]
display_proc(items)
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

実行結果

多くのデータを挿入する
batch_writer を使用

batch_write.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   batch_write.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from decimal import *

# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)
#
with table.batch_writer() as batch:
    min = 30
    for it in range(5):
        timestamp_aa = "2018012208" + str(min) + "00"
        tt = 20.5 + it / 10.0
        temperature = str(tt)
        hh = 31.4 + it / 5.0
        humidity = "%.2f" % hh
        batch.put_item(
            Item={
            'id_device': 'ras0002',
            'timestamp': timestamp_aa,
            'temperature': temperature,
            'humidity': humidity
            }
        )
        min += 1
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

scan を使ってハッシュキーが一致するものを表示
同じことを query を使ってもできます。(query01.py)

scan02.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   scan02.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from boto3.dynamodb.conditions import Key, Attr

# --------------------------------------------------------------------
from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

id_aa = 'ras0002'
sys.stderr.write ("id_aa = " + id_aa + "\n")

response = table.scan(
    FilterExpression=Attr('id_device').eq(id_aa)
    )

display_proc(response['Items'])
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

scan を使ってレンジキーが一致するものを表示

scan03.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   scan03.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from boto3.dynamodb.conditions import Key, Attr

# --------------------------------------------------------------------
from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

timestamp_aa = '20180122082300'
sys.stderr.write ("timestamp_aa = " + timestamp_aa + "\n")

response = table.scan(
    FilterExpression=Attr('timestamp').eq(timestamp_aa)
    )

display_proc(response['Items'])
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

scan を使って、レンジキーが条件を満たすものを表示
この場合は、timestamp がある時間より古いものを表示

scan04.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   scan03.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from boto3.dynamodb.conditions import Key, Attr

# --------------------------------------------------------------------
from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

timestamp_aa = '20180122082300'
sys.stderr.write ("timestamp_aa = " + timestamp_aa + "\n")
#
response = table.scan(
    FilterExpression=Attr('timestamp').lt(timestamp_aa)
    )

display_proc(response['Items'])
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

scan を使って、ハッシュキーが一致して、かつレンジキーが条件を満たすものを表示
この場合は、ras0002 で、timestamp がある時間より古いものを表示

scan05.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   scan05.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from boto3.dynamodb.conditions import Key, Attr

# --------------------------------------------------------------------
from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
table = dynamodb.Table(table_name)

id_aa = 'ras0002'
timestamp_aa = '20180122082300'
sys.stderr.write ("id_aa = " + id_aa + "\n")
sys.stderr.write ("timestamp_aa = " + timestamp_aa + "\n")
#
response = table.scan(
    FilterExpression= Attr('id_device').eq(id_aa) & Attr('timestamp').lt(timestamp_aa))

#
display_proc(response['Items'])
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

batch_get_item　の使い方

batch_get_item.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   batch_get_item.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
# --------------------------------------------------------------------
from display import display_proc
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")

table_name = 'tbl_iot'
#
response = dynamodb.batch_get_item(
    RequestItems={
        table_name: {
            'Keys': [
            {'id_device':'ras0002', 'timestamp':"20180122082200"},
            {'id_device':'ras0002', 'timestamp':"20180122082300"},
            {'id_device':'ras0003', 'timestamp':"20180122082300"}
                ]
            }
    })
#
display_proc(response['Responses'][table_name])
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------

ハッシュキーが一致して、かつレンジキーが条件を満たすものを削除
この場合は、ras0002 で、timestamp がある時間より古いものを削除
batch_writer()　を使用

batch_delete.py

#! /usr/bin/python
# -*- coding: utf-8 -*-
#
#   batch_delete.py
#
#                   Feb/13/2018
# --------------------------------------------------------------------
import  sys
import  boto3
from decimal import *
from boto3.dynamodb.conditions import Key, Attr
# --------------------------------------------------------------------
sys.stderr.write("*** 開始 ***\n")

table_name = 'tbl_iot'
id_aa = 'ras0002'
timestamp_aa = '20180122082300'
sys.stderr.write ("id_aa = " + id_aa + "\n")
sys.stderr.write ("timestamp_aa = " + timestamp_aa + "\n")

dynamodb = boto3.resource('dynamodb',endpoint_url="http://localhost:8000")
client = boto3.client('dynamodb',endpoint_url="http://localhost:8000")
response = client.describe_table(TableName=table_name)
keys = [k['AttributeName'] for k in response['Table']['KeySchema']]
print(keys)

table = dynamodb.Table(table_name)
response = table.scan(
    FilterExpression= Attr('id_device').eq(id_aa) & Attr('timestamp').lt(timestamp_aa))

items = response['Items']
#
with table.batch_writer() as batch:
    for item in items:
        key_dict = {k: item[k] for k in keys}
        print("Deleting " + str(item) + "...")
        print(key_dict)
        batch.delete_item(Key=key_dict)
#
sys.stderr.write("*** 終了 ***\n")
# --------------------------------------------------------------------