Edited at

MacでLocalStackが動かないので応急処置して動かした話

More than 1 year has passed since last update.

MacでLocalStackを動かそうとしましたが、動作しなかったため応急処置をしました。

python3系、python2系双方を検証しました。(dockerでの挙動は未検証です)

なお、以下の内容では、LocalStackのelasticsearchが利用できない可能性があります。


目次


  • 環境

  • python3系


    • とりあえずやってみた

    • 先人の知恵を借りてみた

    • なんとかうごいた



  • python2系


    • とりあえずやってみた

    • elasticsearchを無効化してみた



  • 最後に


環境

OSはmacOS 10.12.5を利用し、pyenvが利用できる状態で環境構築を開始しました。


python3系


とりあえずやってみた

以下のようにインストール及び起動を行ったところ、zipimport.zipimporterに'path'という属性は無いよというエラーが発生しました。

$ pyenv install miniconda3-latest

$ pyenv global miniconda3-latest
$ pip install localstack
Starting local dev environment. CTRL-C to quit.
Error starting infrastructure: 'zipimport.zipimporter' object has no attribute 'path'
Traceback (most recent call last):
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/bin/localstack", line 86, in <module>
infra.start_infra()
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/infra.py", line 362, in start_infra
raise e
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/infra.py", line 306, in start_infra
load_plugins()
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/infra.py", line 100, in load_plugins
file_path = '%s/%s/plugins.py' % (module[0].path, module[1])
AttributeError: 'zipimport.zipimporter' object has no attribute 'path'


先人の知恵を借りてみた

以下の記事で同様の事象が起きていたので、こちらを参考に、infra.pyを編集しました。

http://qiita.com/komeda-shinji/items/88e493601b9827de51da

しかしながら、以下のようなエラーが発生しました。(nodeで発生したエラーは整形しています)

$ vi /Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/infra.py

$ localstack start
Starting local dev environment. CTRL-C to quit.
ERROR: cd "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack" && npm install': b'npm WARN deprecated node-uuid@1.4.8: Use uuid module instead

> leveldown@1.6.0 install /Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/node_modules/leveldown
> prebuild-install || node-gyp rebuild

prebuild-install info begin Prebuild-install version 2.2.1
prebuild-install info looking for local prebuild @ prebuilds/leveldown-v1.6.0-node-v57-darwin-x64.tar.gz
prebuild-install info looking for cached prebuild @ /Users/nogamincho/.npm/_prebuilds/https-github.com-level-leveldown-releases-download-v1.6.0-leveldown-v1.6.0-node-v57-darwin-x64.tar.gz
prebuild-install http request GET https://github.com/level/leveldown/releases/download/v1.6.0/leveldown-v1.6.0-node-v57-darwin-x64.tar.gz
prebuild-install http 404 https://github.com/level/leveldown/releases/download/v1.6.0/leveldown-v1.6.0-node-v57-darwin-x64.tar.gz
prebuild-install WARN install No prebuilt binaries found (target=8.2.1 runtime=node arch=x64 platform=darwin)
gyp ERR! configure error
gyp ERR! stack Error: Command failed: /Users/nogamincho/.pyenv/shims/python2 -c import platform; print(platform.python_version());
gyp ERR! stack pyenv: python2: command not found
gyp ERR! stack
gyp ERR! stack The `python2\' command exists in these Python versions:
gyp ERR! stack anaconda2-4.3.1
gyp ERR! stack miniconda2-latest
gyp ERR! stack
gyp ERR! stack
gyp ERR! stack at ChildProcess.exithandler (child_process.js:270:12)
gyp ERR! stack at emitTwo (events.js:125:13)
gyp ERR! stack at ChildProcess.emit (events.js:213:7)
gyp ERR! stack at maybeClose (internal/child_process.js:921:16)
gyp ERR! stack at Socket.stream.socket.on (internal/child_process.js:348:11)
gyp ERR! stack at emitOne (events.js:115:13)
gyp ERR! stack at Socket.emit (events.js:210:7)
gyp ERR! stack at Pipe._handle.close [as _onclose] (net.js:549:12)
gyp ERR! System Darwin 16.6.0
gyp ERR! command "/usr/local/Cellar/node/8.2.1/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/node_modules/leveldown
gyp ERR! node -v v8.2.1
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm WARN localstack@0.0.1 No repository field.
npm WARN localstack@0.0.1 No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! leveldown@1.6.0 install: `prebuild-install || node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the leveldown@1.6.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/nogamincho/.npm/_logs/2017-07-30T12_33_05_455Z-debug.log

Error starting infrastructure: Command 'cd "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack" && npm install' returned non-zero exit status 1.
Traceback (most recent call last):
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/bin/localstack", line 86, in <module>
infra.start_infra()
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/infra.py", line 363, in start_infra
raise e
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/infra.py", line 329, in start_infra
install.install_components(apis)
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/install.py", line 155, in install_components
parallelize(install_component, names)
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/utils/common.py", line 618, in parallelize
result = pool.map(func, list)
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/multiprocessing/pool.py", line 608, in get
raise self._value
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/install.py", line 147, in install_component
install_kinesalite()
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/services/install.py", line 73, in install_kinesalite
run('cd "%s" && npm install' % ROOT_PATH)
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/utils/common.py", line 521, in run
return do_run(cmd)
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/utils/common.py", line 518, in do_run
raise e
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack/utils/common.py", line 503, in do_run
output = subprocess.check_output(cmd, shell=True, stderr=stderr, env=env_dict, cwd=cwd)
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'cd "/Users/nogamincho/.pyenv/versions/miniconda3-latest/lib/python3.6/site-packages/localstack" && npm install' returned non-zero exit status 1.


なんとかうごいた

python2コマンドをインストールして再実行したところ、一応動作するようになりました。

$ pip install python2

$ localstack start
Starting local dev environment. CTRL-C to quit.
Starting mock ES service (http port 4578)...
Starting local Elasticsearch (http port 4571)...
Starting mock S3 (http port 4572)...
Starting mock SNS (http port 4575)...
Starting mock SQS (http port 4576)...
Starting mock SES (http port 4579)...
Starting mock API Gateway (http port 4567)...
Starting mock DynamoDB (http port 4569)...
Starting mock DynamoDB Streams service (http port 4570)...
Starting mock Firehose service (http port 4573)...
Starting mock Lambda service (http port 4574)...
Starting mock Kinesis (http port 4568)...
Starting mock Redshift (http port 4577)...
Starting mock Route53 (http port 4580)...
Starting mock CloudFormation (http port 4581)...
Starting mock CloudWatch (http port 4582)...
WARNING:infra.py:Service "elasticsearch" not yet available, retrying...
WARNING:infra.py:Service "elasticsearch" not yet available, retrying...
Ready.

$ aws --endpoint-url=http://localhost:4568 kinesis list-streams
{
"StreamNames": []
}

ただし、elasticsearchが正常に動作していない・・・?(elasticsearchは利用していないので、詳細は未確認です)


python2系

python2系でならば問題なく動くのでは・・・?という思いから、


とりあえずやってみた

期待したものの、以下のissueで挙がっているようなエラーが発生しました。

https://github.com/localstack/localstack/issues/213

$ pyenv install miniconda2-latest

$ pyenv global miniconda2-latest
$ pip install localstack
$ localstack start
Starting local dev environment. CTRL-C to quit.
ERROR: 'cd /Users/nogamincho/.pyenv/versions/miniconda2-latest/lib/python2.7/site-packages/localstack/infra && cp /var/folders/_g/2d7f12gj1fq8p666s8sv6_8c0000gn/T/localstack.es.zip es.zip && unzip -q es.zip && mv elasticsearch* elasticsearch && rm es.zip': [es.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of es.zip or
es.zip.zip, and cannot find es.zip.ZIP, period.

Error starting infrastructure: Command 'cd /Users/nogamincho/.pyenv/versions/miniconda2-latest/lib/python2.7/site-packages/localstack/infra && cp /var/folders/_g/2d7f12gj1fq8p666s8sv6_8c0000gn/T/localstack.es.zip es.zip && unzip -q es.zip && mv elasticsearch* elasticsearch && rm es.zip' returned non-zero exit status 9
Traceback (most recent call last):
File "/Users/nogamincho/.pyenv/versions/miniconda2-latest/bin/localstack", line 86, in <module>
infra.start_infra()
File "/Users/nogamincho/.pyenv/versions/miniconda2-latest/lib/python2.7/site-packages/localstack/services/infra.py", line 362, in start_infra
raise e
subprocess32.CalledProcessError: Command 'cd /Users/nogamincho/.pyenv/versions/miniconda2-latest/lib/python2.7/site-packages/localstack/infra && cp /var/folders/_g/2d7f12gj1fq8p666s8sv6_8c0000gn/T/localstack.es.zip es.zip && unzip -q es.zip && mv elasticsearch* elasticsearch && rm es.zip' returned non-zero exit status 9


elasticsearchを無効化してみた

私はelasticsearchを利用する必要がなかったため、力技でelasticsearchを無効化しました。

vi /Users/nogamincho/.pyenv/versions/miniconda2-latest/lib/python2.7/site-packages/localstack/plugins.py

以下の用にlocalstackの起動時に登録するサービスから、elasticsearchを除外しました。

def register_localstack_plugins():

try:
# register_plugin(Plugin('es',
# start=start_elasticsearch_service))
# register_plugin(Plugin('elasticsearch',
# start=es_starter.start_elasticsearch,
# check=es_starter.check_elasticsearch))
register_plugin(Plugin('s3',
start=start_s3,
check=s3_starter.check_s3,
listener=s3_listener.update_s3))

localstackを起動したところ、elasticsearchのサービスは起動せず、エラーが発生しなくなりました。

$ localstack start

Starting local dev environment. CTRL-C to quit.
Starting mock API Gateway (http port 4567)...
Starting mock CloudWatch (http port 4582)...
Starting mock SES (http port 4579)...
Starting mock Kinesis (http port 4568)...
Starting mock Redshift (http port 4577)...
Starting mock S3 (http port 4572)...
Starting mock CloudFormation (http port 4581)...
Starting mock DynamoDB (http port 4569)...
Starting mock SQS (http port 4576)...
Starting mock SNS (http port 4575)...
Starting mock DynamoDB Streams service (http port 4570)...
Starting mock Firehose service (http port 4573)...
Starting mock Route53 (http port 4580)...
Starting mock Lambda service (http port 4574)...
Ready.

$ aws --endpoint-url=http://localhost:4568 kinesis list-streams
{
"StreamNames": []
}


最後に

python3系2系ともに力技で動作させました。

なお、執筆時点(2017/7/30)の情報ですので、最新のbug fixの情報をチェックすることをおすすめします。

最後に補足ですが、バージョンを0.7.2, 0.7.1, 0.6.1.1と落としてみても同様の事象が発生しました。

これはMac側の問題ですかね・・・?