LoginSignup
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

この記事誰得？ 私しか得しないニッチな技術で記事投稿！
Qiita Engineer Festa20242024年7月17日まで開催中！
@Saito5656(5656 Saito)

journalを別logとjsonに出力する

Last updated at Posted at 2024-07-09

この記事について

journalをログに保存するshellscriptの自分用テンプレです。

journalってそのままログ…なんですが別のログにしたいことがあるのでスクリプトを作りました。

別logファイル出力

手元に残るscriptをテンプレ代わりに置きます。ここにgrepでフィルターかけて使うことが多いです。

こんなことしなくても一行で書けるだろう…おっしゃる通りです。ちょっと変形して使ったりできるので一応残してあります。

journal_csv.sh
#!/bin/bash

datetime=$(date '+%Y%m%d-%H%M%S')
# CSVファイルのパス
CSV_FILE="Journal_logs_${datetime}.log"


# journalctl コマンドで最新のログを取得する関数
get_latest_journal() {
    journalctl -n 30 --no-pager
}

# CSVファイルにヘッダーを書き込む（必要なら）
echo "journal" > $CSV_FILE

# 最初のログスナップショットを取得
latest_journal=$(get_latest_journal)

# 無限ループでログの変更を監視
while true; do
    current_journal=$(get_latest_journal)
    if [ "$latest_journal" != "$current_journal" ]; then
        # 一時ファイルに現在のログを一行ずつ追加
        echo "$current_journal" | while IFS= read -r line; do
            # 空行はスキップ
            [ -z "$line" ] && continue
            # 重複行があれば除外
            grep -vFx "$line" $CSV_FILE > temp.csv
            echo "$line" >> temp.csv
            mv temp.csv $CSV_FILE
        done

        latest_journal=$current_journal
    fi
    sleep 0.2
done

普通にこんな感じで出力されます。

json出力

json出力です。
色々と考えることがありjson版を最初に作りました。csv出力はおまけです。

journal_json.sh
#!/bin/bash

THING_NAME=$(uname -n)
# MQTTのパラメータ
MQTT_HOST="127.0.0.1"
MQTT_TOPIC=${THING_NAME}_mos/${THING_NAME}_logs/Journal
echo $MQTT_TOPIC

# journalctl コマンドで最新のログを取得する関数
get_latest_journal() {
    journalctl -n 30 --no-pager
}

# 最初のログスナップショットを取得
latest_journal=$(get_latest_journal)

# 無限ループでログの変更を監視
while true; do
    current_journal=$(get_latest_journal)
    if [ "$latest_journal" != "$current_journal" ]; then
        # 一時ファイルに現在のログを一行ずつ追加
        echo "$current_journal" | while IFS= read -r line; do
            # 空行はスキップ
            [ -z "$line" ] && continue
            # 重複行があれば除外
            if ! grep -Fxq "$line" <<< "$latest_journal"; then
                timestamp=$(date '+%Y-%m-%dT%H:%M:%S')
                json_payload=$(jq -n --arg timestamp "$timestamp" --arg line "$line" '{($timestamp): $line}')
                echo $json_payload
                mosquitto_pub -h "$MQTT_HOST" -t "$MQTT_TOPIC" -m "$json_payload"
            fi
        done

        latest_journal=$current_journal
    fi
    sleep 0.2
done

DEMOとしてlocalhostでpubsubです。

pub
~ $ ./journal_json.sh
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Stopped suricata.service - Suricata IDS/IDP daemon." }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: suricata.service: Consumed 7.155s CPU time." }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Starting suricata.service - Suricata IDS/IDP daemon..." }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: i: suricata: This is Suricata version 7.0.6 RELEASE running in SYSTEM mode" }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: W: ioctl: Failure when trying to get MTU via ioctl for 'eth1': No such device (19)" }
{ "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Started suricata.service - Suricata IDS/IDP daemon." }
{ "2024-07-09T21:44:27": "Jul 09 21:44:26 VNCcamServer1 kernel: brcmfmac mmc1:0001:1 wlan0: entered promiscuous mode" }
{ "2024-07-09T21:44:35": "Jul 09 21:44:35 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54160 DF PROTO=UDP SPT=60161 DPT=20002 LEN=377 " }
{ "2024-07-09T21:44:40": "Jul 09 21:44:39 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54974 DF PROTO=UDP SPT=34506 DPT=20002 LEN=377 " }
{ "2024-07-09T21:45:05": "Jul 09 21:45:05 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=57684 DF PROTO=UDP SPT=51973 DPT=20002 LEN=377 " }
{ "2024-07-09T21:45:10": "Jul 09 21:45:09 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=58091 DF PROTO=UDP SPT=49818 DPT=20002 LEN=377 " }
sub
~ $ mosquitto_sub -h 127.0.0.1 -t "VNCcamServer1_mos/VNCcamServer1_logs/Journal" -v
 "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Stopped suricata.service - Suricata IDS/IDP daemon."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: suricata.service: Consumed 7.155s CPU time."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Starting suricata.service - Suricata IDS/IDP daemon..."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: i: suricata: This is Suricata version 7.0.6 RELEASE running in SYSTEM mode"
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 suricata[24596]: W: ioctl: Failure when trying to get MTU via ioctl for 'eth1': No such device (19)"
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:26": "Jul 09 21:44:25 VNCcamServer1 systemd[1]: Started suricata.service - Suricata IDS/IDP daemon."
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:27": "Jul 09 21:44:26 VNCcamServer1 kernel: brcmfmac mmc1:0001:1 wlan0: entered promiscuous mode"
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:35": "Jul 09 21:44:35 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54160 DF PROTO=UDP SPT=60161 DPT=20002 LEN=377 "
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:44:40": "Jul 09 21:44:39 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=54974 DF PROTO=UDP SPT=34506 DPT=20002 LEN=377 "
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:45:05": "Jul 09 21:45:05 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=57684 DF PROTO=UDP SPT=51973 DPT=20002 LEN=377 "
}
VNCcamServer1_mos/VNCcamServer1_logs/Journal {
  "2024-07-09T21:45:10": "Jul 09 21:45:09 VNCcamServer1 kernel: iptables denied: IN=wlan0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:ae:30:62:62:8d:08:00 SRC=192.168.11.1 DST=192.168.11.255 LEN=397 TOS=0x00 PREC=0x00 TTL=64 ID=58091 DF PROTO=UDP SPT=49818 DPT=20002 LEN=377 "
}
0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?