Docker runner内でRAGを使いたかった
Docker runnerでRAGを使いたかったので、runnerで使うDocker Imageをdind (docker-in-docker) でビルドしようとしたところ、chroma-hnswlib をインストールする部分でエラーが発生しました。
ただ、dindのみでエラーが発生しており、手元のDocker環境では正常にビルドできています。
#9 88.74 Building wheels for collected packages: chroma-hnswlib, pypika, xxxxx
#9 88.74 Building wheel for chroma-hnswlib (pyproject.toml): started
#9 89.11 Building wheel for chroma-hnswlib (pyproject.toml): finished with status 'error'
#9 89.12 error: subprocess-exited-with-error
#9 89.12
#9 89.12 × Building wheel for chroma-hnswlib (pyproject.toml) did not run successfully.
#9 89.12 │ exit code: 1
#9 89.12 ╰─> [60 lines of output]
#9 89.12 running bdist_wheel
#9 89.12 running build
#9 89.12 running build_ext
#9 89.12 creating tmp
#9 89.12 clang -pthread -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -fPIC -fPIC -I/.venv/include -I/opt/rye/py/cpython@3.12.1/include/python3.12 -c /tmp/tmpp6nfpf2f.cpp -o tmp/tmpp6nfpf2f.o -std=c++14
#9 89.12 clang -pthread -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall -fPIC -fPIC -I/.venv/include -I/opt/rye/py/cpython@3.12.1/include/python3.12 -c /tmp/tmp1j63xv2z.cpp -o tmp/tmp1j63xv2z.o -std=c++11
#9 89.12 Traceback (most recent call last):
#9 89.12 File "/tmp/.tmpWqnW88/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#9 89.12 main()
#9 89.12 File "/tmp/.tmpWqnW88/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#9 89.12 json_out['return_val'] = hook(**hook_input['kwargs'])
#9 89.12 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#9 89.12 File "/tmp/.tmpWqnW88/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
#9 89.12 return _build_backend().build_wheel(wheel_directory, config_settings,
#9 89.12 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 410, in build_wheel
#9 89.12 return self._build_with_temp_dir(
#9 89.12 ^^^^^^^^^^^^^^^^^^^^^^^^^^
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 395, in _build_with_temp_dir
#9 89.12 self.run_setup()
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
#9 89.12 exec(code, locals())
#9 89.12 File "<string>", line 115, in <module>
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 104, in setup
#9 89.12 return distutils.core.setup(**attrs)
#9 89.12 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
#9 89.12 return run_commands(dist)
#9 89.12 ^^^^^^^^^^^^^^^^^^
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
#9 89.12 dist.run_commands()
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
#9 89.12 self.run_command(cmd)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
#9 89.12 super().run_command(command)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#9 89.12 cmd_obj.run()
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/wheel/bdist_wheel.py", line 368, in run
#9 89.12 self.run_command("build")
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
#9 89.12 self.distribution.run_command(command)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
#9 89.12 super().run_command(command)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#9 89.12 cmd_obj.run()
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 132, in run
#9 89.12 self.run_command(cmd_name)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
#9 89.12 self.distribution.run_command(command)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 967, in run_command
#9 89.12 super().run_command(command)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
#9 89.12 cmd_obj.run()
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 91, in run
#9 89.12 _build_ext.run(self)
#9 89.12 File "/tmp/pip-build-env-qbf9ww4e/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
#9 89.12 self.build_extensions()
#9 89.12 File "<string>", line 102, in build_extensions
#9 89.12 File "<string>", line 69, in cpp_flag
#9 89.12 RuntimeError: Unsupported compiler -- at least C++11 support is needed!
#9 89.12 [end of output]
#9 89.12
#9 89.12 note: This error originates from a subprocess, and is likely not a problem with pip.
#9 89.12 ERROR: Failed building wheel for chroma-hnswlib
#9 89.13 Building wheel for pypika (pyproject.toml): started
#9 89.46 Building wheel for pypika (pyproject.toml): finished with status 'done'
#9 89.46 Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53724 sha256=fee8a8e0c3f67e19b5416d27f357e54e42003830752f140385221daf9d2a6bb0
#9 89.46 Stored in directory: /root/.cache/pip/wheels/d5/3d/69/8d68d249cd3de2584f226e27fd431d6344f7d70fd856ebd01b
#9 89.46 Building editable for xxxxx (pyproject.toml): started
#9 89.52 Building editable for xxxxx (pyproject.toml): finished with status 'done'
#9 89.52 Created wheel for xxxxx: filename=xxxxx-0.1.0-py3-none-any.whl size=4132 sha256=d0acc4016e7f4eafa6423b64ef91c70946591485c6e3e84ba02584e895bbb12d
#9 89.52 Stored in directory: /tmp/pip-ephem-wheel-cache-6zh5h0fj/wheels/8b/19/c8/73a63a20645e0f1ed9aae9dd5d459f0f7ad2332bb27cba6c0f
#9 89.52 Successfully built pypika xxxxx
#9 89.52 Failed to build chroma-hnswlib
#9 89.52 ERROR: Could not build wheels for chroma-hnswlib, which is required to install pyproject.toml-based projects
ならばと、Dockerfileのベースイメージを FROM gcc:11.4.0 にしましたが改善せず。
最終的に、clang 入れることで解消されました。
FROM gcc:11.4.0
CMD ["/bin/bash"]
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
ENV TZ=Asia/Tokyo
RUN apt-get update && apt-get install -y \
curl \
clang
# rye のインストール
ENV RYE_HOME="/opt/rye"
ENV PATH="$RYE_HOME/shims:$PATH"
RUN curl -sSf https://rye.astral.sh/get | RYE_NO_AUTO_INSTALL=1 RYE_INSTALL_OPTION="--yes" bash
WORKDIR /app
COPY . /app/
RUN rye sync