Cosmos DB に大量にデータを入れてみる

ちょっとした検証目的のため、Cosmos DBに大量データを投じる簡易的なサンプルJavaコードを作成してみました。
当然ながら大量のトランザクションが走ると課金額も増加するため、ご利用は自己責任にてお願いします。

使用方法

1. ニューヨーク市の Taxi and Limousine Commission のページより、Yellowタクシーの乗降データをダウンロードし、任意のフォルダーに配置します。（例：C:\\Temp\\）
2. Eclipseで pom.xml をロードし、Javaパースペクティブを開きます。
3. appsettings.properties を開き、以下のプロパティー値をご自身のCosmos DB環境にあわせて更新します。CsvFileDirには、手順 1) で CSVファイルを配置したフォルダーを指定します。

EndPointUrl=https://REPLACE_ME_WITH_YOURS.documents.azure.com:443/
PrimaryKey=REPLACE_ME_WITH_YOURS
Database=TestDB
Collection=TestCollection
CsvFileDir=C:\\Temp\\

1. GenerateNYCTaxiData.java をJava Applicationとして実行します。

実行例

Created a new database:
{"_rid":"DewXAA==","id":"TestDB","_self":"dbs/DewXAA==/","_colls":"colls/","_users":"users/","_etag":"\"00002908-0000-2300-0000-5e4cf35e0000\"","_ts":1582101343}
Press any key to continue..

Created a new collection:
{"uniqueKeyPolicy":{"uniqueKeys":[]},"geospatialConfig":{"type":"Geography"},"_rid":"DewXAIlOilc=","_sprocs":"sprocs/","_triggers":"triggers/","_udfs":"udfs/","conflictResolutionPolicy":{"mode":"LastWriterWins","conflictResolutionProcedure":"","conflictResolutionPath":"/_ts"},"_docs":"docs/","partitionKey":{"paths":["/_partitionKey"],"kind":"Hash"},"indexingPolicy":{"indexingMode":"consistent","automatic":true,"includedPaths":[{"path":"/*"}],"excludedPaths":[{"path":"/\"_etag\"/?"}]},"id":"TestCollection","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/","_conflicts":"conflicts/","_etag":"\"00001f05-0000-2300-0000-5e4cf3620000\"","_ts":1582101346}
Press any key to continue..
Read a csv file:
C:\Temp\yellow_tripdata_2019-01.csv
Press any key to continue..

Header: VendorID,tpep_pickup_datetime,tpep_dropoff_datetime,passenger_count,trip_distance,RatecodeID,store_and_fwd_flag,PULocationID,DOLocationID,payment_type,fare_amount,extra,mta_tax,tip_amount,tolls_amount,improvement_surcharge,total_amount,congestion_surcharge
Created 1 document:
{"_rid":"DewXAIlOilcBAAAAAAAAAA==","puLocationid":"151","fare_amount":"7","tpep_dropoff_datetime":"2019-01-01 00:53:20","congestion_surcharge":"","vendorid":"1","passenger_count":"1","tolls_amount":"0","improvement_surcharge":"0.3","trip_distance":"1.50","_attachments":"attachments/","store_and_fwd_flag":"N","payment_type":"1","ratecodeid":"1","total_amount":"9.95","extra":"0.5","tip_amount":"1.65","mta_tax":"0.5","id":"92ce47d1-b876-443e-8e47-d1b876643e4a","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/docs/DewXAIlOilcBAAAAAAAAAA==/","doLocationid":"239","tpep_pickup_datetime":"2019-01-01 00:46:40","_etag":"\"00003108-0000-2300-0000-5e4cf3660000\"","_ts":1582101350}
Created 2 document:
{"_rid":"DewXAIlOilcCAAAAAAAAAA==","puLocationid":"239","fare_amount":"14","tpep_dropoff_datetime":"2019-01-01 01:18:59","congestion_surcharge":"","vendorid":"1","passenger_count":"1","tolls_amount":"0","improvement_surcharge":"0.3","trip_distance":"2.60","_attachments":"attachments/","store_and_fwd_flag":"N","payment_type":"1","ratecodeid":"1","total_amount":"16.3","extra":"0.5","tip_amount":"1","mta_tax":"0.5","id":"14a612b4-4eac-4aa3-a612-b44eacdaa3b6","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/docs/DewXAIlOilcCAAAAAAAAAA==/","doLocationid":"246","tpep_pickup_datetime":"2019-01-01 00:59:47","_etag":"\"00003208-0000-2300-0000-5e4cf3660000\"","_ts":1582101350}
Created 3 document:
{"_rid":"DewXAIlOilcDAAAAAAAAAA==","puLocationid":"236","fare_amount":"4.5","tpep_dropoff_datetime":"2018-12-21 13:52:40","congestion_surcharge":"","vendorid":"2","passenger_count":"3","tolls_amount":"0","improvement_surcharge":"0.3","trip_distance":".00","_attachments":"attachments/","store_and_fwd_flag":"N","payment_type":"1","ratecodeid":"1","total_amount":"5.8","extra":"0.5","tip_amount":"0","mta_tax":"0.5","id":"69d7e305-53e1-4efd-97e3-0553e18efd36","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/docs/DewXAIlOilcDAAAAAAAAAA==/","doLocationid":"236","tpep_pickup_datetime":"2018-12-21 13:48:30","_etag":"\"00003308-0000-2300-0000-5e4cf3660000\"","_ts":1582101350}
Created 4 document:
{"_rid":"DewXAIlOilcEAAAAAAAAAA==","puLocationid":"193","fare_amount":"3.5","tpep_dropoff_datetime":"2018-11-28 15:55:45","congestion_surcharge":"","vendorid":"2","passenger_count":"5","tolls_amount":"0","improvement_surcharge":"0.3","trip_distance":".00","_attachments":"attachments/","store_and_fwd_flag":"N","payment_type":"2","ratecodeid":"1","total_amount":"7.55","extra":"0.5","tip_amount":"0","mta_tax":"0.5","id":"3cb3e88a-da63-4060-b3e8-8ada63d060b3","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/docs/DewXAIlOilcEAAAAAAAAAA==/","doLocationid":"193","tpep_pickup_datetime":"2018-11-28 15:52:25","_etag":"\"00003408-0000-2300-0000-5e4cf3660000\"","_ts":1582101350}
Created 5 document:
{"_rid":"DewXAIlOilcFAAAAAAAAAA==","puLocationid":"193","fare_amount":"52","tpep_dropoff_datetime":"2018-11-28 15:58:33","congestion_surcharge":"","vendorid":"2","passenger_count":"5","tolls_amount":"0","improvement_surcharge":"0.3","trip_distance":".00","_attachments":"attachments/","store_and_fwd_flag":"N","payment_type":"2","ratecodeid":"2","total_amount":"55.55","extra":"0","tip_amount":"0","mta_tax":"0.5","id":"ff1002e6-03f5-4bac-9002-e603f5cbacf5","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/docs/DewXAIlOilcFAAAAAAAAAA==/","doLocationid":"193","tpep_pickup_datetime":"2018-11-28 15:56:57","_etag":"\"00003508-0000-2300-0000-5e4cf3660000\"","_ts":1582101350}
Created 6 document:
{"_rid":"DewXAIlOilcGAAAAAAAAAA==","puLocationid":"193","fare_amount":"3.5","tpep_dropoff_datetime":"2018-11-28 16:28:26","congestion_surcharge":"","vendorid":"2","passenger_count":"5","tolls_amount":"5.76","improvement_surcharge":"0.3","trip_distance":".00","_attachments":"attachments/","store_and_fwd_flag":"N","payment_type":"2","ratecodeid":"1","total_amount":"13.31","extra":"0.5","tip_amount":"0","mta_tax":"0.5","id":"3cf17f8b-617e-49a4-b17f-8b617e29a49c","_self":"dbs/DewXAA==/colls/DewXAIlOilc=/docs/DewXAIlOilcGAAAAAAAAAA==/","doLocationid":"193","tpep_pickup_datetime":"2018-11-28 16:25:49","_etag":"\"00003608-0000-2300-0000-5e4cf3660000\"","_ts":1582101350}
Created 7 document:

:

参考情報

• azure-documentdb-java (GitHub)