概要
scikit-learn インストール時に Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx などとエラーが発生しており、対策がどこにも見当たらなかったので、記事化しました。
発生していたエラー
全文は長すぎるので、一部のみ表示しています。
#0 38.57 File "/tmp/pip-install-eejs_jf6/scikit-learn_06e4db7e458a4de681aab31b38e24e2c/sklearn/_build_utils/__init__.py", line 98, in cythonize_extensions
#0 38.57 return cythonize(
#0 38.57 ^^^^^^^^^^
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1136, in cythonize
#0 38.57 result.get(99999) # seconds
#0 38.57 ^^^^^^^^^^^^^^^^^
#0 38.57 File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
#0 38.57 raise self._value
#0 38.57 Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx
#0 38.57 [end of output]
エラーを確認
#0 38.57 warning: /tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/numpy/__init__.cython-30.pxd:1046:48: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
#0 38.57
#0 38.57 Error compiling Cython file:
#0 38.57 ------------------------------------------------------------
#0 38.57 ...
#0 38.57
#0 38.57 from libc.math cimport fabs, sqrt
#0 38.57 cimport numpy as cnp
#0 38.57 import numpy as np
#0 38.57 from cython cimport floating
#0 38.57 from numpy.math cimport isnan
#0 38.57 ^
#0 38.57 ------------------------------------------------------------
... 略
#0 38.57 Error compiling Cython file:
#0 38.57 ------------------------------------------------------------
#0 38.57 ...
#0 38.57 means[i] /= sum_weights[i]
#0 38.57
#0 38.57 for col_ind in range(n_features):
#0 38.57 for i in range(X_indptr[col_ind], X_indptr[col_ind + 1]):
#0 38.57 row_ind = X_indices[i]
#0 38.57 if not isnan(X_data[i]):
#0 38.57 ^
#0 38.57 ------------------------------------------------------------
#0 38.57
#0 38.57 sklearn/utils/sparsefuncs_fast.pyx:273:19: 'isnan' is not a constant, variable or function identifier
#0 38.57 Traceback (most recent call last):
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1297, in cythonize_one_helper
#0 38.57 return cythonize_one(*m)
#0 38.57 ^^^^^^^^^^^^^^^^^
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1289, in cythonize_one
#0 38.57 raise CompileError(None, pyx_file)
#0 38.57 Cython.Compiler.Errors.CompileError: sklearn/utils/sparsefuncs_fast.pyx
#0 38.57 multiprocessing.pool.RemoteTraceback:
#0 38.57 """
#0 38.57 Traceback (most recent call last):
#0 38.57 File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
#0 38.57 result = (True, func(*args, **kwds))
#0 38.57 ^^^^^^^^^^^^^^^^^^^
#0 38.57 File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 48, in mapstar
#0 38.57 return list(map(*args))
#0 38.57 ^^^^^^^^^^^^^^^^
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1297, in cythonize_one_helper
#0 38.57 return cythonize_one(*m)
#0 38.57 ^^^^^^^^^^^^^^^^^
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1289, in cythonize_one
#0 38.57 raise CompileError(None, pyx_file)
#0 38.57 Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx
#0 38.57 """
#0 38.57
#0 38.57 The above exception was the direct cause of the following exception:
#0 38.57
#0 38.57 Traceback (most recent call last):
#0 38.57 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
#0 38.57 [ 1/64] Cythonizing sklearn/__check_build/_check_build.pyx
#0 38.57 [ 2/64] Cythonizing sklearn/_isotonic.pyx
#0 38.57 [ 3/64] Cythonizing sklearn/_loss/_loss.pyx
#0 38.57 [ 4/64] Cythonizing sklearn/cluster/_dbscan_inner.pyx
#0 38.57 [ 5/64] Cythonizing sklearn/cluster/_hierarchical_fast.pyx
#0 38.57 [ 6/64] Cythonizing sklearn/cluster/_k_means_common.pyx
#0 38.57 [ 7/64] Cythonizing sklearn/cluster/_k_means_elkan.pyx
#0 38.57 [ 8/64] Cythonizing sklearn/cluster/_k_means_lloyd.pyx
#0 38.57 [ 9/64] Cythonizing sklearn/cluster/_k_means_minibatch.pyx
#0 38.57 [10/64] Cythonizing sklearn/datasets/_svmlight_format_fast.pyx
#0 38.57 [11/64] Cythonizing sklearn/decomposition/_cdnmf_fast.pyx
#0 38.57 [12/64] Cythonizing sklearn/decomposition/_online_lda_fast.pyx
#0 38.57 [13/64] Cythonizing sklearn/ensemble/_gradient_boosting.pyx
#0 38.57 [14/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_binning.pyx
#0 38.57 [15/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_bitset.pyx
#0 38.57 [16/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_gradient_boosting.pyx
#0 38.57 [17/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_predictor.pyx
#0 38.57 [18/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/common.pyx
#0 38.57 [19/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/histogram.pyx
#0 38.57 [20/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/splitting.pyx
#0 38.57 [21/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/utils.pyx
#0 38.57 [22/64] Cythonizing sklearn/feature_extraction/_hashing_fast.pyx
#0 38.57 [23/64] Cythonizing sklearn/linear_model/_cd_fast.pyx
#0 38.57 [24/64] Cythonizing sklearn/linear_model/_sag_fast.pyx
#0 38.57 [25/64] Cythonizing sklearn/linear_model/_sgd_fast.pyx
#0 38.57 [26/64] Cythonizing sklearn/manifold/_barnes_hut_tsne.pyx
#0 38.57 [27/64] Cythonizing sklearn/manifold/_utils.pyx
#0 38.57 [28/64] Cythonizing sklearn/metrics/_dist_metrics.pyx
#0 38.57 [29/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_argkmin.pyx
#0 38.57 [30/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_base.pyx
#0 38.57 [31/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.pyx
#0 38.57 [32/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx
#0 38.57 [33/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx
#0 38.57 [34/64] Cythonizing sklearn/metrics/_pairwise_fast.pyx
#0 38.57 [35/64] Cythonizing sklearn/metrics/cluster/_expected_mutual_info_fast.pyx
#0 38.57 [36/64] Cythonizing sklearn/neighbors/_ball_tree.pyx
#0 38.57 [37/64] Cythonizing sklearn/neighbors/_kd_tree.pyx
#0 38.57 [38/64] Cythonizing sklearn/neighbors/_partition_nodes.pyx
#0 38.57 [39/64] Cythonizing sklearn/neighbors/_quad_tree.pyx
#0 38.57 [40/64] Cythonizing sklearn/preprocessing/_csr_polynomial_expansion.pyx
#0 38.57 [41/64] Cythonizing sklearn/svm/_liblinear.pyx
#0 38.57 [42/64] Cythonizing sklearn/svm/_libsvm.pyx
#0 38.57 [43/64] Cythonizing sklearn/svm/_libsvm_sparse.pyx
#0 38.57 [44/64] Cythonizing sklearn/svm/_newrand.pyx
#0 38.57 [45/64] Cythonizing sklearn/tree/_criterion.pyx
#0 38.57 [46/64] Cythonizing sklearn/tree/_splitter.pyx
#0 38.57 [47/64] Cythonizing sklearn/tree/_tree.pyx
#0 38.57 [48/64] Cythonizing sklearn/tree/_utils.pyx
#0 38.57 [49/64] Cythonizing sklearn/utils/_cython_blas.pyx
#0 38.57 [50/64] Cythonizing sklearn/utils/_fast_dict.pyx
#0 38.57 [51/64] Cythonizing sklearn/utils/_heap.pyx
#0 38.57 [52/64] Cythonizing sklearn/utils/_isfinite.pyx
#0 38.57 [53/64] Cythonizing sklearn/utils/_logistic_sigmoid.pyx
#0 38.57 [54/64] Cythonizing sklearn/utils/_openmp_helpers.pyx
#0 38.57 [55/64] Cythonizing sklearn/utils/_random.pyx
#0 38.57 [56/64] Cythonizing sklearn/utils/_readonly_array_wrapper.pyx
#0 38.57 [57/64] Cythonizing sklearn/utils/_seq_dataset.pyx
#0 38.57 [58/64] Cythonizing sklearn/utils/_sorting.pyx
#0 38.57 [59/64] Cythonizing sklearn/utils/_typedefs.pyx
#0 38.57 [60/64] Cythonizing sklearn/utils/_vector_sentinel.pyx
#0 38.57 [61/64] Cythonizing sklearn/utils/_weight_vector.pyx
#0 38.57 [62/64] Cythonizing sklearn/utils/arrayfuncs.pyx
#0 38.57 [63/64] Cythonizing sklearn/utils/murmurhash.pyx
#0 38.57 [64/64] Cythonizing sklearn/utils/sparsefuncs_fast.pyx
#0 38.57 main()
#0 38.57 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
#0 38.57 json_out["return_val"] = hook(**hook_input["kwargs"])
#0 38.57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#0 38.57 File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 175, in prepare_metadata_for_build_wheel
#0 38.57 return hook(metadata_directory, config_settings)
#0 38.57 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 374, in prepare_metadata_for_build_wheel
#0 38.57 self.run_setup()
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 512, in run_setup
#0 38.57 super().run_setup(setup_script=setup_script)
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 317, in run_setup
#0 38.57 exec(code, locals())
#0 38.57 File "<string>", line 669, in <module>
#0 38.57 File "<string>", line 663, in setup_package
#0 38.57 File "<string>", line 597, in configure_extension_modules
#0 38.57 File "/tmp/pip-install-eejs_jf6/scikit-learn_06e4db7e458a4de681aab31b38e24e2c/sklearn/_build_utils/__init__.py", line 98, in cythonize_extensions
#0 38.57 return cythonize(
#0 38.57 ^^^^^^^^^^
#0 38.57 File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1136, in cythonize
#0 38.57 result.get(99999) # seconds
#0 38.57 ^^^^^^^^^^^^^^^^^
#0 38.57 File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
#0 38.57 raise self._value
#0 38.57 Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx
#0 38.57 [end of output]
#0 38.57
#0 38.57 note: This error originates from a subprocess, and is likely not a problem with pip.
#0 38.57 error: metadata-generation-failed
#0 38.57
#0 38.57 × Encountered error while generating package metadata.
#0 38.57 ╰─> See above for output.
#0 38.57
#0 38.57 note: This is an issue with the package mentioned above, not pip.
#0 38.57 hint: See above for details.
------
failed to solve: executor failed running [/bin/sh -c pip install -r requirements.txt]: exit code: 1
実行環境
以下の環境で発生し、対処によってエラーを解消させました
| Item | version |
|---|---|
| Docker image | python:3.12.10-slim |
| python | 3.12.10 |
| Cython | 3.1.0 |
| scikit-learn | 1.2.2 |
| scikit-learn | 1.6.1 (version up してもダメ) |
対処法
Cython のバージョンを 3 未満にすることで解決しました。
具体的には Cython==0.29.35 を requirements.txt に記載し、そのほかは上記実行環境のバージョンのままで、インストールに成功しました。
参考になれば嬉しいです(*˘︶˘*).｡.:*
参考にした記事
私が遭遇したエラーとは別のエラーが発生しているようですが、このメッセージにヒントを得て、利用する Cython のバージョンを下げてみることにしました。
ちなみに、見てみて驚いたのですが、 Cython って 3.0.0 の手前はいきなり 0.29.36 なんですね。
2.x.x とかは無いみたいです( ˘ω˘)ｽﾔｧ...
気になる方は、以下を参照してみてください。