1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

M1 Macでscrapy shellでfetchしたらMemory Error

Posted at

タイトルの通り、Scrapyを使ってみようとしたら、こんなエラーが出た


In [1]: fetch('https://qiita.com/')
2024-05-01 16:50:02 [scrapy.core.engine] INFO: Spider opened
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 fetch('https://qiita.com/')

File /opt/anaconda3/envs/scrapy_env/lib/python3.8/site-packages/scrapy/shell.py:110, in Shell.fetch(self, request_or_url, spider, redirect, **kwargs)
    108 response = None
    109 try:
--> 110     response, spider = threads.blockingCallFromThread(
    111         reactor, self._schedule, request, spider)
    112 except IgnoreRequest:
    113     pass

File /opt/anaconda3/envs/scrapy_env/lib/python3.8/site-packages/twisted/internet/threads.py:120, in blockingCallFromThread(reactor, f, *a, **kw)
    118 result = queue.get()
    119 if isinstance(result, failure.Failure):
--> 120     result.raiseException()
    121 return result

File /opt/anaconda3/envs/scrapy_env/lib/python3.8/site-packages/twisted/python/failure.py:504, in Failure.raiseException(self)
    499 def raiseException(self) -> NoReturn:
    500     """
    501     raise the original exception, preserving traceback
    502     information if available.
    503     """
--> 504     raise self.value.with_traceback(self.tb)

MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks

M1 Macで起こるらしい、cffiのせいらしい

下記で解決

$ conda uninstall --force cffi
$ conda install cffi

参照URL
https://stackoverflow.com/questions/67556847/scrapy-on-m1-mac-memoryerror-cannot-allocate-writeexecute-memory-for-ffi-call

1
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?