KinesisAgentインストールと動作確認

KinesisFirehoseを使うのでProducerにKinesisエージェントを使ってみる。メモとして記録。検証時のOSはAmazon Linux 2。

AmazonLinux方式のインストール

$ sudo yum update
$ sudo yum install aws-kinesis-agent
Loaded plugins: langpacks, priorities, update-motd
104 packages excluded due to repository priority protections
No package aws-kinesis-agent available.

AmazonLinux方式うまくいかない。。
調査する余裕もないのでS3にあるほうをインストールする

$ sudo yum install –y https://s3.amazonaws.com/streaming-data-agent/aws-kinesis-agent-latest.amzn1.noarch.rpm
======================================================================================================================
 Package                        Arch      Version                     Repository                                 Size
======================================================================================================================
Installing:
 aws-kinesis-agent              noarch    1.1.3-1.amzn1               /aws-kinesis-agent-latest.amzn1.noarch     18 M
Installing for dependencies:
 alsa-lib                       x86_64    1.1.3-3.amzn2               amzn2-core                                425 k
 copy-jdk-configs               noarch    2.2-3.amzn2                 amzn2-core                                 19 k
 geronimo-jms                   noarch    1.1.1-19.amzn2              amzn2-core                                 31 k
 giflib                         x86_64    4.1.6-9.amzn2               amzn2-core                                 40 k
 java-1.8.0-openjdk             x86_64    1:1.8.0.161-0.b14.amzn2     amzn2-core                                242 k
 java-1.8.0-openjdk-headless    x86_64    1:1.8.0.161-0.b14.amzn2     amzn2-core                                 32 M
 javamail                       noarch    1.4.6-8.amzn2               amzn2-core                                757 k
 javapackages-tools             noarch    3.4.1-11.amzn2              amzn2-core                                 73 k
 libXcomposite                  x86_64    0.4.4-4.1.amzn2             amzn2-core                                 22 k
 libXfont                       x86_64    1.5.2-1.amzn2               amzn2-core                                152 k
 libXi                          x86_64    1.7.9-1.amzn2               amzn2-core                                 41 k
 libXrender                     x86_64    0.9.10-1.amzn2              amzn2-core                                 26 k
 libXtst                        x86_64    1.2.3-1.amzn2               amzn2-core                                 20 k
 libfontenc                     x86_64    1.1.3-3.amzn2               amzn2-core                                 31 k
 lksctp-tools                   x86_64    1.0.17-2.amzn2              amzn2-core                                 88 k
 log4j                          noarch    1.2.17-16.amzn2             amzn2-core                                444 k
 python-javapackages            noarch    3.4.1-11.amzn2              amzn2-core                                 31 k
 python-lxml                    x86_64    3.2.1-4.amzn2               amzn2-core                                779 k
 ttmkfdir                       x86_64    3.0.9-42.amzn2              amzn2-core                                 50 k
 tzdata-java                    noarch    2018c-1.amzn2               amzn2-core                                184 k
 xorg-x11-font-utils            x86_64    1:7.5-20.amzn2              amzn2-core                                 87 k
 xorg-x11-fonts-Type1           noarch    7.5-9.amzn2                 amzn2-core                                521 k

Transaction Summary
======================================================================================================================
Install  1 Package (+22 Dependent packages)

設定ファイルを確認。デフォルトだとこんな状態。

$ ls -l /etc/aws-kinesis/agent.json
-rw-r--r-- 1 root root 338 Dec 22 09:17 /etc/aws-kinesis/agent.json
$ cat /etc/aws-kinesis/agent.json
{
  "cloudwatch.emitMetrics": true,
  "kinesis.endpoint": "",
  "firehose.endpoint": "",

  "flows": [
    {
      "filePattern": "/tmp/app.log*",
      "kinesisStream": "yourkinesisstream",
      "partitionKeyOption": "RANDOM"
    },
    {
      "filePattern": "/tmp/app.log*",
      "deliveryStream": "yourdeliverystream"
    }
  ]
}

  • エージェントがデータを集めるファイル ("filePattern")
  • エージェントがデータを送信するストリーム ("kinesisStream")
    Kinesis側の設定が完了してから設定する

  • 自動起動を有効にする

$ sudo chkconfig aws-kinesis-agent on
  • 手動で起動する
$ sudo service aws-kinesis-agent start
Starting aws-kinesis-agent (via systemctl):                [  OK  ]
  • ログファイルが作られているので確認
$ cat /var/log/aws-kinesis-agent/aws-kinesis-agent.log
Sign up for free and join this conversation.
Sign Up
If you already have a Qiita account log in.