Go to Qiita Advent Calendar 2024 Top
LoginSignup
60
65

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@nw-engineerin株式会社ライブエスト

Pythonのセキュリティ関連ライブラリを試してみました。

Last updated at Posted at 2024-08-08

セキュリティが重要視される昨今、開発したプログラムの脆弱性を検出・適切な対策を講じることは不可欠だと思います。（というか業務であれやこれやと言われるわけでして。）そこでPythonを使ったアプリ開発が多い私としましても、役に立つツールがあればよいなと思い、いくつかのツールを試してみました。検証したツールは以下4つとなります。

  • bandit
  • pylint
  • safety
  • Snyk(Pythonに限らず広く利用可能)

前提

検査に利用したファイルは以下記事でご紹介したmain.pyファイルとなります。また環境も同じ環境を利用しています。

1. bandit

Banditは、Pythonコードのセキュリティ脆弱性を静的解析によって検出するオープンソースツール。特定のセキュリティリスクを識別し、修正を手助けしてくれるツールのようです。

以下インストールから実行の結果となります。

# pip3 install bandit
# bandit main.py
[main]  INFO    profile include tests: None
[main]  INFO    profile exclude tests: None
[main]  INFO    cli include tests: None
[main]  INFO    cli exclude tests: None
[main]  INFO    running on Python 3.9.2
Run started:2024-08-08 04:56:06.213410

Test results:
        No issues identified.

Code scanned:
        Total lines of code: 71
        Total lines skipped (#nosec): 0

Run metrics:
        Total issues (by severity):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 0
        Total issues (by confidence):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 0
Files skipped (0):

各項目の意味

項目 内容 結果
Test results 今回のテスト結果 問題なし
Total lines of code スキャンされたコードの行数 71
Total lines skipped スキップされた行数 0
Low 重大度(低) 0
Medium 重大度(中) 0
High 重大度(大) 0
Low 信頼度(低) 0
Medium 信頼度(中) 0
High 信頼度(大) 0
Files skipped スキップされたファイルのリスト 0

今回は問題なしと判定されていますが、役に立ちそうなので常に利用し続けたいですね。

公式ページ

2. pylint

PyLintはコードの品質とセキュリティを向上させるための静的解析ツール。スタイルガイドに従ったコードを書きやすくし、潜在的なバグを早期発見に役立つようです。

インストール・実行結果

# pip3 install pylint
# pylint main.py
************* Module main
main.py:10:0: C0301: Line too long (114/100) (line-too-long)
main.py:97:0: C0305: Trailing newlines (trailing-newlines)
main.py:1:0: C0114: Missing module docstring (missing-module-docstring)
main.py:28:0: C0115: Missing class docstring (missing-class-docstring)
main.py:32:0: C0116: Missing function or method docstring (missing-function-docstring)
main.py:37:0: C0116: Missing function or method docstring (missing-function-docstring)
main.py:59:11: W0718: Catching too general exception Exception (broad-exception-caught)
main.py:64:0: C0116: Missing function or method docstring (missing-function-docstring)
main.py:3:0: C0411: third party import "pydantic.BaseModel" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:4:0: C0411: standard import "sqlite3" should be placed before third party import "pydantic.BaseModel" and first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:5:0: C0411: third party import "openai" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:6:0: C0411: third party import "sklearn.feature_extraction.text.TfidfVectorizer" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:7:0: C0411: third party import "sklearn.cluster.KMeans" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:8:0: C0411: third party import "numpy" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:1:0: W0611: Unused HTTPException imported from fastapi (unused-import)
main.py:8:0: W0611: Unused numpy imported as np (unused-import)

------------------------------------------------------------------
Your code has been rated at 7.09/10 (previous run: 7.09/10, +0.00)

コードに対していくつかのスタイルや慣行に関する警告、エラー報告が出ていました。
7.09/10の評価と表示されました。改善の余地がありますが、大きな問題は少ないという事だと思います。

それぞれ意味

main.py:10:0: C0301: Line too long (114/100) (line-too-long)

10行目が長いので100文字以内にしなさい

main.py:97:0: C0305: Trailing newlines (trailing-newlines)

97行目に余分な改行がある

main.py:1:0: C0114: Missing module docstring (missing-module-docstring)
main.py:28:0: C0115: Missing class docstring (missing-class-docstring)
main.py:32:0: C0116: Missing function or method docstring (missing-function-docstring)
main.py:37:0: C0116: Missing function or method docstring (missing-function-docstring)
main.py:64:0: C0116: Missing function or method docstring (missing-function-docstring)

モジュール、クラス、関数に対してコメントがない。

main.py:59:11: W0718: Catching too general exception Exception (broad-exception-caught)

具体的な例外を捕捉するように修正しない。

main.py:3:0: C0411: third party import "pydantic.BaseModel" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:4:0: C0411: standard import "sqlite3" should be placed before third party import "pydantic.BaseModel" and first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:5:0: C0411: third party import "openai" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:6:0: C0411: third party import "sklearn.feature_extraction.text.TfidfVectorizer" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:7:0: C0411: third party import "sklearn.cluster.KMeans" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)
main.py:8:0: C0411: third party import "numpy" should be placed before first party imports "fastapi.FastAPI", "fastapi.middleware.cors.CORSMiddleware"  (wrong-import-order)

インポート順序が違う。

main.py:1:0: W0611: Unused HTTPException imported from fastapi (unused-import)
main.py:8:0: W0611: Unused numpy imported as np (unused-import)

使用されていないモジュールがある。

・・・・・たんまりご指摘を受けました。

改修できるポイントはしていきたいと思います。修正後のコードをいかに示します。

main.py
"""
FastAPI/OpenAIモジュール読込
"""
import sqlite3
from pydantic import BaseModel
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
import openai
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans

openai.api_key = (
    'sk-xx_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
    'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
)

app = FastAPI()

app.add_middleware(
    CORSMiddleware,
    allow_origins=['*'],
    allow_credentials=True,
    allow_methods=['*'],
    allow_headers=['*'],
)

# SQLite3 データベースのセットアップ
conn = sqlite3.connect('ranking.db', check_same_thread=False)
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS entries (id INTEGER PRIMARY KEY, text TEXT)''')
conn.commit()

class InputData(BaseModel):
    """
    Reactから渡されるInputデータ用クラス
    """
    text: str

@app.post("/submit/")
async def submit(data: InputData):
    """
    データ挿入用関数
    """
    c.execute("INSERT INTO entries (text) VALUES (?)", (data.text,))
    conn.commit()
    return {"message": "Successfully"}

def group_similar_texts(texts, num_clusters):
    """
    クラスタリング用関数
    """
    try:
        vectorizer = TfidfVectorizer().fit_transform(texts)
        vectors = vectorizer.toarray()

        kmeans = KMeans(n_clusters=num_clusters, random_state=0).fit(vectors)

        clusters = {}
        for idx, label in enumerate(kmeans.labels_):
            if label not in clusters:
                clusters[label] = []
            clusters[label].append(texts[idx])

        return list(clusters.values())

    except Exception as e:
        print(f"Error in group_similar_texts: {e}")
        return []

@app.get("/rankings/")
async def get_rankings():
    """
    ランキング用関数
    """
    c.execute("SELECT text FROM entries")
    rows = c.fetchall()
    texts = [row[0] for row in rows]

    if not texts:
        return {"rankings": []}

    grouped_texts = group_similar_texts(texts, num_clusters=5)

    summaries = []
    for group in grouped_texts:
        combined_text = " ".join(group)
        response = openai.ChatCompletion.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a helpful assistant."},
                {"role": "user", "content": f"次の文章を要約して下さい: {combined_text}"}
            ],
            max_tokens=100,
            temperature=0.0
        )
        summary = response.choices[0].message['content'].strip()
        summaries.append((combined_text, summary, len(group)))

    sorted_summaries = sorted(summaries, key=lambda x: x[2], reverse=True)

    rankings = [{"combined_text": s[0], "summary": s[1], "count": s[2]} for s in sorted_summaries]

    return {"rankings": rankings}

再度実行

# pylint main.py
************* Module main
/opt/fastapi/main.py:66:11: W0718: Catching too general exception Exception (broad-exception-caught)

------------------------------------------------------------------
Your code has been rated at 9.80/10 (previous run: 9.80/10, +0.00)

限りなく100%に近づきました。(例外処理はすみませんが割愛します)

公式ページ

3. safety

Safetyは、Pythonパッケージの依存関係に対するセキュリティ脆弱性をチェックするツールです。既知の脆弱性を持つパッケージを検出し、アップデートや修正を促します。

インストール/実行結果

# pip3 install safety
# safety check
+===================================================================================================================================================================================+

                               /$$$$$$            /$$
                              /$$__  $$          | $$
           /$$$$$$$  /$$$$$$ | $$  \__//$$$$$$  /$$$$$$   /$$   /$$
          /$$_____/ |____  $$| $$$$   /$$__  $$|_  $$_/  | $$  | $$
         |  $$$$$$   /$$$$$$$| $$_/  | $$$$$$$$  | $$    | $$  | $$
          \____  $$ /$$__  $$| $$    | $$_____/  | $$ /$$| $$  | $$
          /$$$$$$$/|  $$$$$$$| $$    |  $$$$$$$  |  $$$$/|  $$$$$$$
         |_______/  \_______/|__/     \_______/   \___/   \____  $$
                                                          /$$  | $$
                                                         |  $$$$$$/
  by safetycli.com                                        \______/

+===================================================================================================================================================================================+

 REPORT

  Safety is using PyUp's free open-source vulnerability database. This data is 30 days old and limited.
  For real-time enhanced vulnerability data, fix recommendations, severity reporting, cybersecurity support, team and project policy management and more sign up at
https://pyup.io or email sales@pyup.io

  Safety v3.2.4 is scanning for Vulnerabilities...
  Scanning dependencies in your environment:

  -> /usr/local/lib/python3.9/dist-packages
  -> /usr/lib/python39.zip
  -> /usr/lib/python3.9
  -> /usr/local/bin
  -> /usr/lib/python3/dist-packages
  -> /usr/lib/python3.9/lib-dynload
  -> /usr/local/lib/python3.9/dist-packages/setuptools/_vendor

  Using open-source vulnerability database
  Found and scanned 85 packages
  Timestamp 2024-08-08 04:58:21
  6 vulnerabilities reported
  0 vulnerabilities ignored

+===================================================================================================================================================================================+
 VULNERABILITIES REPORTED
+===================================================================================================================================================================================+

-> Vulnerability found in wheel version 0.34.2
   Vulnerability ID: 51499
   Affected spec: <0.38.1
   ADVISORY: Wheel 0.38.1 includes a fix for CVE-2022-40898: An issue discovered in Python Packaging Authority (PyPA) Wheel 0.37.1 and earlier allows remote attackers to
   cause a denial of service via attacker controlled input to wheel cli.https://pyup.io/posts/pyup-discovers-redos-vulnerabilities-in-top-python-packages
   CVE-2022-40898
   For more information about this vulnerability, visit https://data.safetycli.com/v/51499/97c
   To ignore this vulnerability, use PyUp vulnerability id 51499 in safety’s ignore command-line argument or add the ignore to your safety policy file.


-> Vulnerability found in pip version 20.3.4
   Vulnerability ID: 62044
   Affected spec: <23.3
   ADVISORY: Affected versions of Pip are vulnerable to Command Injection. When installing a package from a Mercurial VCS URL (ie "pip install hg+...") with pip prior to
   v23.3, the specified Mercurial revision could be used to inject arbitrary configuration options to the "hg clone" call (ie "--config"). Controlling the Mercurial...
   CVE-2023-5752
   For more information about this vulnerability, visit https://data.safetycli.com/v/62044/97c
   To ignore this vulnerability, use PyUp vulnerability id 62044 in safety’s ignore command-line argument or add the ignore to your safety policy file.


-> Vulnerability found in pip version 20.3.4
   Vulnerability ID: 67599
   Affected spec: <21.1
   ADVISORY: An issue was discovered in Pip (all versions) because it installs the version with the highest version number, even if the user had intended to obtain a
   private package from a private index. This only affects use of the --extra-index-url option, and exploitation requires that the package does not already exist in the public...
   CVE-2018-20225
   For more information about this vulnerability, visit https://data.safetycli.com/v/67599/97c
   To ignore this vulnerability, use PyUp vulnerability id 67599 in safety’s ignore command-line argument or add the ignore to your safety policy file.


-> Vulnerability found in pip version 20.3.4
   Vulnerability ID: 42559
   Affected spec: <21.1
   ADVISORY: A flaw was found in python-pip in the way it handled Unicode separators in git references. A remote attacker could possibly use this issue to install a
   different revision on a repository. The highest threat from this vulnerability is to data integrity.
   CVE-2021-3572
   For more information about this vulnerability, visit https://data.safetycli.com/v/42559/97c
   To ignore this vulnerability, use PyUp vulnerability id 42559 in safety’s ignore command-line argument or add the ignore to your safety policy file.


-> Vulnerability found in py version 1.11.0
   Vulnerability ID: 51457
   Affected spec: <=1.11.0
   ADVISORY: Py throughout 1.11.0 allows remote attackers to conduct a ReDoS (Regular expression Denial of Service) attack via a Subversion repository with crafted info
   data, because the InfoSvnCommand argument is mishandled.https://github.com/pytest-dev/py/issues/287
   CVE-2022-42969
   For more information about this vulnerability, visit https://data.safetycli.com/v/51457/97c
   To ignore this vulnerability, use PyUp vulnerability id 51457 in safety’s ignore command-line argument or add the ignore to your safety policy file.


-> Vulnerability found in jinja2 version 3.1.4
   Vulnerability ID: 70612
   Affected spec: >=0
   ADVISORY: In Jinja2, the from_string function is prone to Server Side Template Injection (SSTI) where it takes the "source" parameter as a template object, renders it,
   and then returns it. The attacker can exploit it with INJECTION COMMANDS in a URI. NOTE: The maintainer and multiple third parties believe that this vulnerability isn't...
   CVE-2019-8341
   For more information about this vulnerability, visit https://data.safetycli.com/v/70612/97c
   To ignore this vulnerability, use PyUp vulnerability id 70612 in safety’s ignore command-line argument or add the ignore to your safety policy file.


+===================================================================================================================================================================================+
   REMEDIATIONS

  6 vulnerabilities were reported in 4 packages. For detailed remediation & fix recommendations, upgrade to a commercial license.

+===================================================================================================================================================================================+

 Scan was completed. 6 vulnerabilities were reported.

+===================================================================================================================================================================================+

  Safety is using PyUp's free open-source vulnerability database. This data is 30 days old and limited.
  For real-time enhanced vulnerability data, fix recommendations, severity reporting, cybersecurity support, team and project policy management and more sign up at
https://pyup.io or email sales@pyup.io

+===================================================================================================================================================================================

いくつかの依存関係に対するセキュリティ脆弱性が報告されています。脆弱性を修正するためそれぞれのバージョンを上げたいと思います。

pip3 install --upgrade wheel
pip3 install --upgrade pip

※pyとJinja2は2024年8月8日現在、pypi.orgで確認する限りインストールされているバージョンが最新だったため、上記二つのみ実行しています。

再実行

root@7a8bcc68d4a4:/# safety check
+================================================================================================================================================================+

                               /$$$$$$            /$$
                              /$$__  $$          | $$
           /$$$$$$$  /$$$$$$ | $$  \__//$$$$$$  /$$$$$$   /$$   /$$
          /$$_____/ |____  $$| $$$$   /$$__  $$|_  $$_/  | $$  | $$
         |  $$$$$$   /$$$$$$$| $$_/  | $$$$$$$$  | $$    | $$  | $$
          \____  $$ /$$__  $$| $$    | $$_____/  | $$ /$$| $$  | $$
          /$$$$$$$/|  $$$$$$$| $$    |  $$$$$$$  |  $$$$/|  $$$$$$$
         |_______/  \_______/|__/     \_______/   \___/   \____  $$
                                                          /$$  | $$
                                                         |  $$$$$$/
  by safetycli.com                                        \______/

+================================================================================================================================================================+

 REPORT

  Safety is using PyUp's free open-source vulnerability database. This data is 30 days old and limited.
  For real-time enhanced vulnerability data, fix recommendations, severity reporting, cybersecurity support, team and project policy management and more sign
up at https://pyup.io or email sales@pyup.io

  Safety v3.2.4 is scanning for Vulnerabilities...
  Scanning dependencies in your environment:

  -> /usr/lib/python3/dist-packages
  -> /usr/local/lib/python3.9/dist-packages
  -> /usr/lib/python3.9
  -> /usr/local/lib/python3.9/dist-packages/setuptools/_vendor
  -> /usr/lib/python39.zip
  -> /usr/local/bin
  -> /usr/lib/python3.9/lib-dynload

  Using open-source vulnerability database
  Found and scanned 89 packages
  Timestamp 2024-08-08 07:07:01
  2 vulnerabilities reported
  0 vulnerabilities ignored

+================================================================================================================================================================+
 VULNERABILITIES REPORTED
+================================================================================================================================================================+

-> Vulnerability found in py version 1.11.0
   Vulnerability ID: 51457
   Affected spec: <=1.11.0
   ADVISORY: Py throughout 1.11.0 allows remote attackers to conduct a ReDoS (Regular expression Denial of Service) attack via a Subversion repository
   with crafted info data, because the InfoSvnCommand argument is mishandled.https://github.com/pytest-dev/py/issues/287
   CVE-2022-42969
   For more information about this vulnerability, visit https://data.safetycli.com/v/51457/97c
   To ignore this vulnerability, use PyUp vulnerability id 51457 in safety’s ignore command-line argument or add the ignore to your safety policy file.


-> Vulnerability found in jinja2 version 3.1.4
   Vulnerability ID: 70612
   Affected spec: >=0
   ADVISORY: In Jinja2, the from_string function is prone to Server Side Template Injection (SSTI) where it takes the "source" parameter as a template
   object, renders it, and then returns it. The attacker can exploit it with INJECTION COMMANDS in a URI. NOTE: The maintainer and multiple third parties...
   CVE-2019-8341
   For more information about this vulnerability, visit https://data.safetycli.com/v/70612/97c
   To ignore this vulnerability, use PyUp vulnerability id 70612 in safety’s ignore command-line argument or add the ignore to your safety policy file.


+================================================================================================================================================================+
   REMEDIATIONS

  2 vulnerabilities were reported in 2 packages. For detailed remediation & fix recommendations, upgrade to a commercial license.

+================================================================================================================================================================+

 Scan was completed. 2 vulnerabilities were reported.

+================================================================================================================================================================+

  Safety is using PyUp's free open-source vulnerability database. This data is 30 days old and limited.
  For real-time enhanced vulnerability data, fix recommendations, severity reporting, cybersecurity support, team and project policy management and more sign
up at https://pyup.io or email sales@pyup.io

+================================================================================================================================================================+

対応していない二つのみとなりました。

公式ページ

4. Snyk

Snykは、依存関係の脆弱性を検出、修正方法を提案するクラウドベースのセキュリティツールです。継続的なセキュリティ監視を提供、プロジェクトの安全性を高めてくれます！
※言語はPythonに限りません！今回はGithubにあるリポジトリをチェックしたいと思います。

無料で利用できる範囲のテストとなりますが、以下サイトにアクセスしましょう。

「Snyk」を使ってみるをクリックします。
image.png

画面が遷移したら「Github」をクリックします。
image.png

ID/Passをいれて、「Sign in」をクリックします。
image.png

アプリで認証しましょう。
image.png

連携してよいか聞かれますので、「Authorize snyk」をクリックします。
image.png

それでは同じmain.pyを取り込んだリポジトリを確認しましょう。「view project」をクリックします。
image.png

1件、ハードコードされた認証情報ある旨の通知が出ています。
image.png

5.その他

利用方法はどれもそれほど難しいわけではないため、それぞれセルフチェック用途に利用するとよいなと感じました。

60
65
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
60
65

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?