タイトルの通り、Scrapyを使ってみようとしたら、こんなエラーが出た
In [1]: fetch('https://qiita.com/')
2024-05-01 16:50:02 [scrapy.core.engine] INFO: Spider opened
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
Cell In[1], line 1
----> 1 fetch('https://qiita.com/')
File /opt/anaconda3/envs/scrapy_env/lib/python3.8/site-packages/scrapy/shell.py:110, in Shell.fetch(self, request_or_url, spider, redirect, **kwargs)
108 response = None
109 try:
--> 110 response, spider = threads.blockingCallFromThread(
111 reactor, self._schedule, request, spider)
112 except IgnoreRequest:
113 pass
File /opt/anaconda3/envs/scrapy_env/lib/python3.8/site-packages/twisted/internet/threads.py:120, in blockingCallFromThread(reactor, f, *a, **kw)
118 result = queue.get()
119 if isinstance(result, failure.Failure):
--> 120 result.raiseException()
121 return result
File /opt/anaconda3/envs/scrapy_env/lib/python3.8/site-packages/twisted/python/failure.py:504, in Failure.raiseException(self)
499 def raiseException(self) -> NoReturn:
500 """
501 raise the original exception, preserving traceback
502 information if available.
503 """
--> 504 raise self.value.with_traceback(self.tb)
MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks
M1 Macで起こるらしい、cffiのせいらしい
下記で解決
$ conda uninstall --force cffi
$ conda install cffi