LoginSignup
2
1

More than 5 years have passed since last update.

@selfnavi(YAMAMOTO Takashi)

Real-time Packet Observation Tool (RPOT) を ubuntu 18.04 で動かす

Last updated at Posted at 2019-01-10

Real-time Packet Observation Tool をインストール

概要

Real-time Packet Observation Tool (RPOT) の README.md をみつつインストールします。本家は適当に端折られていますが、下記はインストール過程に使うコマンドをすべて記録しています（作業途中の出力については適宜端折っています）。

環境

解析する pcap ファイルの規模によると思いますが、超絶重いアプリケーションです。

  • ubuntu 18.04
  • core-i5 メモリ 8G

インストール

  1. docker-compose と必要なパッケージをインストール

    apt_install_docker-compose
    [yamachan@ubuntu ~]$ sudo su
[sudo] yamachan のパスワード:
[root@ubuntu yamachan]# apt update
- 略 -
[root@ubuntu yamachan]# apt upgrade
- 略 -
続行しますか? [Y/n] y
- 略 -
[root@ubuntu yamachan]# apt install docker-compose
パッケージリストを読み込んでいます... 完了
依存関係ツリーを作成しています
状態情報を読み取っています... 完了
以下の追加パッケージがインストールされます:
bridge-utils cgroupfs-mount docker.io golang-docker-credential-helpers pigz python-asn1crypto python-backports.ssl-match-hostname python-cached-property
python-certifi python-cffi-backend python-chardet python-cryptography python-docker python-dockerpty python-dockerpycreds python-docopt python-enum34
python-funcsigs python-functools32 python-idna python-ipaddress python-jsonschema python-mock python-openssl python-pbr python-pkg-resources python-requests
python-six python-texttable python-urllib3 python-websocket python-yaml ubuntu-fan
提案パッケージ:
aufs-tools btrfs-progs debootstrap docker-doc rinse python-cryptography-doc python-cryptography-vectors python-enum34-doc python-funcsigs-doc python-mock-doc
python-openssl-doc python-openssl-dbg python-setuptools python-socks python-ntlm
以下のパッケージが新たにインストールされます:
bridge-utils cgroupfs-mount docker-compose docker.io golang-docker-credential-helpers pigz python-asn1crypto python-backports.ssl-match-hostname
python-cached-property python-certifi python-cffi-backend python-chardet python-cryptography python-docker python-dockerpty python-dockerpycreds python-docopt
python-enum34 python-funcsigs python-functools32 python-idna python-ipaddress python-jsonschema python-mock python-openssl python-pbr python-pkg-resources
python-requests python-six python-texttable python-urllib3 python-websocket python-yaml ubuntu-fan
アップグレード: 0 個、新規インストール: 34 個、削除: 0 個、保留: 0 個。
42.3 MB のアーカイブを取得する必要があります。
この操作後に追加で 207 MB のディスク容量が消費されます。
続行しますか? [Y/n] y
略
[root@ubuntu yamachan]# exit
exit

  2. README.md のとおりスレッドの上限をあげる

    max_map_count
    [yamachan@ubuntu ~]$ cat /proc/sys/vm/max_map_count
65530
[yamachan@ubuntu ~]$ echo 'vm.max_map_count = 262144' | sudo tee -a /etc/sysctl.conf
[sudo] yamachan のパスワード:
vm.max_map_count = 262144
[yamachan@ubuntu ~]$ sudo sysctl -p
vm.max_map_count = 262144
[yamachan@ubuntu ~]$ cat /proc/sys/vm/max_map_count
262144

  3. ここで docker-compose pull を実行しても失敗します。あたりまえか。

    git_clone
    [yamachan@ubuntu ~]$ docker-compose pull
ERROR:
    Can't find a suitable configuration file in this directory or any
    parent. Are you in the right directory?

    Supported filenames: docker-compose.yml, docker-compose.yaml

  4. git レポジトリより、rpot を clone します。

  5. clone したら、docker-compose pull を実行するも失敗、なぜだ。

    git_clone
    [yamachan@ubuntu ~]$ git clone https://github.com/super-a1ice/rpot.git
Cloning into 'rpot'...
remote: Enumerating objects: 308, done.
remote: Total 308 (delta 0), reused 0 (delta 0), pack-reused 308
Receiving objects: 100% (308/308), 25.44 MiB | 1.92 MiB/s, done.
Resolving deltas: 100% (129/129), done.
　
[yamachan@ubuntu ~]$ cd rpot/
[yamachan@ubuntu rpot]$ docker-compose pull
Pulling zookeeper (wurstmeister/zookeeper:latest)...
ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?
　
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

  6. この問題の解決には、単に docker グループに自身を加えればいいという記事が多い中 If you faced an issue like “Couldn’t connect to Docker daemon at http+docker://localunixsocket — is it running?”… が親切に説明しているのでその通りに調べてみる

    • まずは docker daemon が動いているかどうか、どうやらまともに動いているっぽい
    Check_docker_operation
    [yamachan@ubuntu rpot]$ sudo service docker status
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
Active: active (running) since Thu 2019-01-10 10:05:33 JST; 19min ago
 Docs: https://docs.docker.com
Main PID: 26791 (dockerd)
Tasks: 30
CGroup: /system.slice/docker.service
       tq26791 /usr/bin/dockerd -H fd://
       mq26815 docker-containerd --config /var/run/docker/containerd/containerd.toml
　
1月 10 10:05:31 ubuntu dockerd[26791]: time="2019-01-10T10:05:31.803620849+09:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42023
1月 10 10:05:31 ubuntu dockerd[26791]: time="2019-01-10T10:05:31.803871119+09:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc42023
1月 10 10:05:31 ubuntu dockerd[26791]: time="2019-01-10T10:05:31.803955441+09:00" level=info msg="Loading containers: start."
1月 10 10:05:32 ubuntu dockerd[26791]: time="2019-01-10T10:05:32.523214514+09:00" level=info msg="Default bridge (docker0) is assigned with an IP addre
1月 10 10:05:32 ubuntu dockerd[26791]: time="2019-01-10T10:05:32.920223182+09:00" level=info msg="Loading containers: done."
1月 10 10:05:32 ubuntu dockerd[26791]: time="2019-01-10T10:05:32.985440077+09:00" level=warning msg="failed to retrieve docker-runc version: unknown ou
1月 10 10:05:32 ubuntu dockerd[26791]: time="2019-01-10T10:05:32.989692423+09:00" level=info msg="Docker daemon" commit=e68fc7a graphdriver(s)=zfs vers
1月 10 10:05:32 ubuntu dockerd[26791]: time="2019-01-10T10:05:32.992411718+09:00" level=info msg="Daemon has completed initialization"
1月 10 10:05:33 ubuntu dockerd[26791]: time="2019-01-10T10:05:33.160966540+09:00" level=info msg="API listen on /var/run/docker.sock"
1月 10 10:05:33 ubuntu systemd[1]: Started Docker Application Container Engine.
    • 次に socket の権限、自身（yamachan）に権限がないので加える、グループを有効にするためシェルから抜けます
    Add_to_docker_group
    [yamachan@ubuntu rpot]$ sudo ls -la /var/run/docker.sock
srw-rw---- 1 root docker 0  1月 10 10:05 /var/run/docker.sock
[yamachan@ubuntu rpot]$ sudo usermod -aG docker ${USER}
[yamachan@ubuntu rpot]$ exit
exit

  7. 再びログインして、docker グループに自身（yamachan）が加わっていることを確認して、

  8. docker-compose pull （数百メガ引っ張ってきますので待ちます）

    docker-compose_pull
    [yamachan@ubuntu ~]$ id
uid=1000(yamachan) gid=1000(yamachan) groups=1000(yamachan),4(adm),20(dialout),24(cdrom),27(sudo),30(dip),46(plugdev),116(lpadmin),126(sambashare),127(vboxusers),133(docker),10000(admin),10001(public)
[yamachan@ubuntu ~]$ cd rpot/
[yamachan@ubuntu rpot]$ docker-compose pull
Pulling zookeeper (wurstmeister/zookeeper:latest)...
latest: Pulling from wurstmeister/zookeeper
略
Digest: sha256:f8122897f0a30b314234151789cc4b69dc579762ee9a380faa83b67a4b5bad99
Status: Downloaded newer image for tatsui/bro:latest

  9. docker-compose up manager （立ち上がるまで 90 秒くらいかかります）

    docker-compose_up_manager
    [yamachan@ubuntu rpot]$ docker-compose up manager
elasticsearch is up-to-date
rpot_zookeeper_1 is up-to-date
rpot_kafka_1 is up-to-date
rpot_logstash_1 is up-to-date
Creating rpot_kibana_1 ...
Creating rpot_kibana_1 ... done
Creating rpot_manager_1 ...
Creating rpot_manager_1 ... done
Attaching to rpot_manager_1
略
manager_1        | {"acknowledged":true}rpot_manager_1 exited with code 0
[yamachan@ubuntu rpot]$

  10. docker-compose up bro ここでは解析対象 pcap ファイルがないと怒られます。

    • そういえば README.md に step 1 copy or mount pcap file directory とかいてありましたね... しかしコピー先の pcap ディレクトリが git clone のときはなかったけど
    docker-compose_up_bro
    [yamachan@ubuntu rpot]$ docker-compose up bro
elasticsearch is up-to-date
rpot_zookeeper_1 is up-to-date
rpot_kafka_1 is up-to-date
rpot_logstash_1 is up-to-date
Creating rpot_bro_1 ...
Creating rpot_bro_1 ... done
Attaching to rpot_bro_1
bro_1            | ls: cannot access '*.pcap': No such file or directory
rpot_bro_1 exited with code 0

  11. 見てみると、docker-compose up bro 実行時に pcap ディレクトリが作られている

    Put_a_pcap_file_1
    [yamachan@ubuntu rpot]$ ls -la
合計 114
drwxrwxr-x 13 yamachan yamachan    20  1月 10 11:24 .
drwxr-xr-x 51 yamachan yamachan    77  1月 10 11:15 ..
drwxrwxr-x  8 yamachan yamachan    13  1月 10 10:15 .git
-rw-rw-r--  1 yamachan yamachan    64  1月 10 10:15 .gitignore
-rw-rw-r--  1 yamachan yamachan    87  1月 10 10:15 .gitmodules
-rw-rw-r--  1 yamachan yamachan 11337  1月 10 10:15 LICENSE
-rw-rw-r--  1 yamachan yamachan  3129  1月 10 10:15 README.md
drwxrwxr-x 11 yamachan yamachan    11  1月 10 10:15 antivirus
drwxrwxr-x  3 yamachan yamachan     5  1月 10 10:15 bro
drwxrwxr-x  3 yamachan yamachan     3  1月 10 10:15 doc
-rw-rw-r--  1 yamachan yamachan  2034  1月 10 10:15 docker-compose-hunting.yml
-rw-rw-r--  1 yamachan yamachan  4161  1月 10 10:15 docker-compose-scale.yml
-rw-rw-r--  1 yamachan yamachan  2507  1月 10 10:15 docker-compose.yml
drwxr-xr-x  2 root     root         2  1月 10 11:24 extract_files
drwxrwxr-x  3 yamachan yamachan     9  1月 10 10:15 logstash
drwxrwxr-x  4 yamachan yamachan     7  1月 10 10:15 manager
drwxr-xr-x  2 root     root         2  1月 10 11:24 pcap
drwxrwxr-x  4 yamachan yamachan     8  1月 10 10:15 suricata
drwxrwxr-x  3 yamachan yamachan     5  1月 10 10:15 yara
drwxrwxr-x  2 yamachan yamachan     5  1月 10 10:15 yara-gen

  12. pcap ディレクトリに権限がないので変更し、

  13. あらかじめ用意していた解析用ファイル（example.pcap）を pcap ディレクトリに入れ、

  14. docker-compose up bro で起動します

    Put_a_pcap_file_2
    [yamachan@ubuntu rpot]$ sudo chown -R yamachan:yamachan pcap extract_files
[yamachan@ubuntu rpot]$ mv ~/example.pcap pcap
[yamachan@ubuntu rpot]$ docker-compose up bro
rpot_zookeeper_1 is up-to-date
elasticsearch is up-to-date
rpot_kafka_1 is up-to-date
rpot_logstash_1 is up-to-date
Starting rpot_bro_1 ...
Starting rpot_bro_1 ... done
Attaching to rpot_bro_1
bro_1            | scan example.pcap standard mode
bro_1            | packet_filter/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | loaded_scripts/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593781.729090 reporter/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593781.729090 stats/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593784.054183 weird/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593787.078383 conn/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593791.991236 dns/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593828.634234 files/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593828.634234 http/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | 1544593838.324624 capture_loss/Log::WRITER_KAFKAWRITER: Debug is turned off.
bro_1            | WARNING: No Site::local_nets have been defined.  It's usually a good idea to define your local networks.
rpot_bro_1 exited with code 0

  15. README.md にはlocalhost をブラウザで見ろ（ttp://localhost:5601）とありますが localhost 以外でも見れないかどう確認します

    • どこからでも OK なようです。
    Check_access_permission
    [yamachan@ubuntu rpot]$ sudo lsof -i | grep 5601
docker-pr 26590            root    4u  IPv6  66894      0t0  TCP *:5601 (LISTEN)

  16. 現在の IP アドレスを調べてブラウザから ttp://192.168.1.199:5601（私の場合）にアクセスします

    [yamachan@ubuntu rpot]$ ifconfig | grep -1 eno1
　
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.199  netmask 255.255.255.0  broadcast 192.168.1.255

  17. kibana の画面が出てきます

  18. とめます

    docker-compose_down
    [yamachan@ubuntu rpot]$ docker-compose down -v
Stopping rpot_logstash_1  ... done
Stopping rpot_kafka_1     ... done
Stopping rpot_kibana_1    ... done
Stopping rpot_zookeeper_1 ... done
Stopping elasticsearch    ... done
Removing rpot_bro_1       ... done
Removing rpot_manager_1   ... done
Removing rpot_logstash_1  ... done
Removing rpot_kafka_1     ... done
Removing rpot_kibana_1    ... done
Removing rpot_zookeeper_1 ... done
Removing elasticsearch    ... done
Removing network rpot_frontend
Removing network rpot_backend
Removing volume rpot_rules-data
Removing volume rpot_json-data
Removing volume rpot_es-data
2
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
2
1