概要

scikit-learn インストール時に Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx などとエラーが発生しており、対策がどこにも見当たらなかったので、記事化しました。

発生していたエラー

全文は長すぎるので、一部のみ表示しています。

#0 38.57         File "/tmp/pip-install-eejs_jf6/scikit-learn_06e4db7e458a4de681aab31b38e24e2c/sklearn/_build_utils/__init__.py", line 98, in cythonize_extensions
#0 38.57           return cythonize(
#0 38.57                  ^^^^^^^^^^
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1136, in cythonize
#0 38.57           result.get(99999)  # seconds
#0 38.57           ^^^^^^^^^^^^^^^^^
#0 38.57         File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
#0 38.57           raise self._value
#0 38.57       Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx
#0 38.57       [end of output]

エラーを確認

#0 38.57       warning: /tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/numpy/__init__.cython-30.pxd:1046:48: Implicit noexcept declaration is deprecated. Function declaration should contain 'noexcept' keyword.
#0 38.57       
#0 38.57       Error compiling Cython file:
#0 38.57       ------------------------------------------------------------
#0 38.57       ...
#0 38.57       
#0 38.57       from libc.math cimport fabs, sqrt
#0 38.57       cimport numpy as cnp
#0 38.57       import numpy as np
#0 38.57       from cython cimport floating
#0 38.57       from numpy.math cimport isnan
#0 38.57       ^
#0 38.57       ------------------------------------------------------------

... 略

#0 38.57       Error compiling Cython file:
#0 38.57       ------------------------------------------------------------
#0 38.57       ...
#0 38.57               means[i] /= sum_weights[i]
#0 38.57       
#0 38.57           for col_ind in range(n_features):
#0 38.57               for i in range(X_indptr[col_ind], X_indptr[col_ind + 1]):
#0 38.57                   row_ind = X_indices[i]
#0 38.57                   if not isnan(X_data[i]):
#0 38.57                          ^
#0 38.57       ------------------------------------------------------------
#0 38.57       
#0 38.57       sklearn/utils/sparsefuncs_fast.pyx:273:19: 'isnan' is not a constant, variable or function identifier
#0 38.57       Traceback (most recent call last):
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1297, in cythonize_one_helper
#0 38.57           return cythonize_one(*m)
#0 38.57                  ^^^^^^^^^^^^^^^^^
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1289, in cythonize_one
#0 38.57           raise CompileError(None, pyx_file)
#0 38.57       Cython.Compiler.Errors.CompileError: sklearn/utils/sparsefuncs_fast.pyx
#0 38.57       multiprocessing.pool.RemoteTraceback:
#0 38.57       """
#0 38.57       Traceback (most recent call last):
#0 38.57         File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 125, in worker
#0 38.57           result = (True, func(*args, **kwds))
#0 38.57                           ^^^^^^^^^^^^^^^^^^^
#0 38.57         File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 48, in mapstar
#0 38.57           return list(map(*args))
#0 38.57                  ^^^^^^^^^^^^^^^^
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1297, in cythonize_one_helper
#0 38.57           return cythonize_one(*m)
#0 38.57                  ^^^^^^^^^^^^^^^^^
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1289, in cythonize_one
#0 38.57           raise CompileError(None, pyx_file)
#0 38.57       Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx
#0 38.57       """
#0 38.57       
#0 38.57       The above exception was the direct cause of the following exception:
#0 38.57       
#0 38.57       Traceback (most recent call last):
#0 38.57         File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
#0 38.57       [ 1/64] Cythonizing sklearn/__check_build/_check_build.pyx
#0 38.57       [ 2/64] Cythonizing sklearn/_isotonic.pyx
#0 38.57       [ 3/64] Cythonizing sklearn/_loss/_loss.pyx
#0 38.57       [ 4/64] Cythonizing sklearn/cluster/_dbscan_inner.pyx
#0 38.57       [ 5/64] Cythonizing sklearn/cluster/_hierarchical_fast.pyx
#0 38.57       [ 6/64] Cythonizing sklearn/cluster/_k_means_common.pyx
#0 38.57       [ 7/64] Cythonizing sklearn/cluster/_k_means_elkan.pyx
#0 38.57       [ 8/64] Cythonizing sklearn/cluster/_k_means_lloyd.pyx
#0 38.57       [ 9/64] Cythonizing sklearn/cluster/_k_means_minibatch.pyx
#0 38.57       [10/64] Cythonizing sklearn/datasets/_svmlight_format_fast.pyx
#0 38.57       [11/64] Cythonizing sklearn/decomposition/_cdnmf_fast.pyx
#0 38.57       [12/64] Cythonizing sklearn/decomposition/_online_lda_fast.pyx
#0 38.57       [13/64] Cythonizing sklearn/ensemble/_gradient_boosting.pyx
#0 38.57       [14/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_binning.pyx
#0 38.57       [15/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_bitset.pyx
#0 38.57       [16/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_gradient_boosting.pyx
#0 38.57       [17/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/_predictor.pyx
#0 38.57       [18/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/common.pyx
#0 38.57       [19/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/histogram.pyx
#0 38.57       [20/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/splitting.pyx
#0 38.57       [21/64] Cythonizing sklearn/ensemble/_hist_gradient_boosting/utils.pyx
#0 38.57       [22/64] Cythonizing sklearn/feature_extraction/_hashing_fast.pyx
#0 38.57       [23/64] Cythonizing sklearn/linear_model/_cd_fast.pyx
#0 38.57       [24/64] Cythonizing sklearn/linear_model/_sag_fast.pyx
#0 38.57       [25/64] Cythonizing sklearn/linear_model/_sgd_fast.pyx
#0 38.57       [26/64] Cythonizing sklearn/manifold/_barnes_hut_tsne.pyx
#0 38.57       [27/64] Cythonizing sklearn/manifold/_utils.pyx
#0 38.57       [28/64] Cythonizing sklearn/metrics/_dist_metrics.pyx
#0 38.57       [29/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_argkmin.pyx
#0 38.57       [30/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_base.pyx
#0 38.57       [31/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_datasets_pair.pyx
#0 38.57       [32/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_middle_term_computer.pyx
#0 38.57       [33/64] Cythonizing sklearn/metrics/_pairwise_distances_reduction/_radius_neighbors.pyx
#0 38.57       [34/64] Cythonizing sklearn/metrics/_pairwise_fast.pyx
#0 38.57       [35/64] Cythonizing sklearn/metrics/cluster/_expected_mutual_info_fast.pyx
#0 38.57       [36/64] Cythonizing sklearn/neighbors/_ball_tree.pyx
#0 38.57       [37/64] Cythonizing sklearn/neighbors/_kd_tree.pyx
#0 38.57       [38/64] Cythonizing sklearn/neighbors/_partition_nodes.pyx
#0 38.57       [39/64] Cythonizing sklearn/neighbors/_quad_tree.pyx
#0 38.57       [40/64] Cythonizing sklearn/preprocessing/_csr_polynomial_expansion.pyx
#0 38.57       [41/64] Cythonizing sklearn/svm/_liblinear.pyx
#0 38.57       [42/64] Cythonizing sklearn/svm/_libsvm.pyx
#0 38.57       [43/64] Cythonizing sklearn/svm/_libsvm_sparse.pyx
#0 38.57       [44/64] Cythonizing sklearn/svm/_newrand.pyx
#0 38.57       [45/64] Cythonizing sklearn/tree/_criterion.pyx
#0 38.57       [46/64] Cythonizing sklearn/tree/_splitter.pyx
#0 38.57       [47/64] Cythonizing sklearn/tree/_tree.pyx
#0 38.57       [48/64] Cythonizing sklearn/tree/_utils.pyx
#0 38.57       [49/64] Cythonizing sklearn/utils/_cython_blas.pyx
#0 38.57       [50/64] Cythonizing sklearn/utils/_fast_dict.pyx
#0 38.57       [51/64] Cythonizing sklearn/utils/_heap.pyx
#0 38.57       [52/64] Cythonizing sklearn/utils/_isfinite.pyx
#0 38.57       [53/64] Cythonizing sklearn/utils/_logistic_sigmoid.pyx
#0 38.57       [54/64] Cythonizing sklearn/utils/_openmp_helpers.pyx
#0 38.57       [55/64] Cythonizing sklearn/utils/_random.pyx
#0 38.57       [56/64] Cythonizing sklearn/utils/_readonly_array_wrapper.pyx
#0 38.57       [57/64] Cythonizing sklearn/utils/_seq_dataset.pyx
#0 38.57       [58/64] Cythonizing sklearn/utils/_sorting.pyx
#0 38.57       [59/64] Cythonizing sklearn/utils/_typedefs.pyx
#0 38.57       [60/64] Cythonizing sklearn/utils/_vector_sentinel.pyx
#0 38.57       [61/64] Cythonizing sklearn/utils/_weight_vector.pyx
#0 38.57       [62/64] Cythonizing sklearn/utils/arrayfuncs.pyx
#0 38.57       [63/64] Cythonizing sklearn/utils/murmurhash.pyx
#0 38.57       [64/64] Cythonizing sklearn/utils/sparsefuncs_fast.pyx
#0 38.57           main()
#0 38.57         File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
#0 38.57           json_out["return_val"] = hook(**hook_input["kwargs"])
#0 38.57                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#0 38.57         File "/usr/local/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 175, in prepare_metadata_for_build_wheel
#0 38.57           return hook(metadata_directory, config_settings)
#0 38.57                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 374, in prepare_metadata_for_build_wheel
#0 38.57           self.run_setup()
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 512, in run_setup
#0 38.57           super().run_setup(setup_script=setup_script)
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 317, in run_setup
#0 38.57           exec(code, locals())
#0 38.57         File "<string>", line 669, in <module>
#0 38.57         File "<string>", line 663, in setup_package
#0 38.57         File "<string>", line 597, in configure_extension_modules
#0 38.57         File "/tmp/pip-install-eejs_jf6/scikit-learn_06e4db7e458a4de681aab31b38e24e2c/sklearn/_build_utils/__init__.py", line 98, in cythonize_extensions
#0 38.57           return cythonize(
#0 38.57                  ^^^^^^^^^^
#0 38.57         File "/tmp/pip-build-env-7ofe86g4/overlay/lib/python3.12/site-packages/Cython/Build/Dependencies.py", line 1136, in cythonize
#0 38.57           result.get(99999)  # seconds
#0 38.57           ^^^^^^^^^^^^^^^^^
#0 38.57         File "/usr/local/lib/python3.12/multiprocessing/pool.py", line 774, in get
#0 38.57           raise self._value
#0 38.57       Cython.Compiler.Errors.CompileError: sklearn/cluster/_hierarchical_fast.pyx
#0 38.57       [end of output]
#0 38.57   
#0 38.57   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 38.57 error: metadata-generation-failed
#0 38.57 
#0 38.57 × Encountered error while generating package metadata.
#0 38.57 ╰─> See above for output.
#0 38.57 
#0 38.57 note: This is an issue with the package mentioned above, not pip.
#0 38.57 hint: See above for details.
------
failed to solve: executor failed running [/bin/sh -c pip install -r requirements.txt]: exit code: 1

実行環境

以下の環境で発生し、対処によってエラーを解消させました

Item	version
Docker image	python:3.12.10-slim
python	3.12.10
Cython	3.1.0
scikit-learn	1.2.2
scikit-learn	1.6.1 (version up してもダメ)

対処法

Cython のバージョンを 3 未満にすることで解決しました。

具体的には Cython==0.29.35 を requirements.txt に記載し、そのほかは上記実行環境のバージョンのままで、インストールに成功しました。

参考になれば嬉しいです(*˘︶˘*).｡.:*

参考にした記事

私が遭遇したエラーとは別のエラーが発生しているようですが、このメッセージにヒントを得て、利用する Cython のバージョンを下げてみることにしました。

ちなみに、見てみて驚いたのですが、 Cython って 3.0.0 の手前はいきなり 0.29.36 なんですね。

2.x.x とかは無いみたいです( ˘ω˘)ｽﾔｧ...

気になる方は、以下を参照してみてください。