aws s3 syncをしようとしたときに、--excludeを大量に書きたくないので、excludeしたいファイルを別ファイルに書いて、シェルスクリプト内で読み込んでで実行するようにしたところ、*(アスタリスク)が自動展開されて困ったので対応のメモ
除外ファイル一覧
.s3_sync_exclude
Gemfile
Gemfile.lock
.git/*
.gitignore
最初に作ったシェルスクリプト
sync_dryrun.sh
#!/bin/bash
dir=$(cd $(dirname ${0}); pwd)
file=${0##*/}
command="aws s3 sync ${dir}/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude ${file}"
while read exclude
do
command="${command} --exclude ${exclude}"
done < ${dir}/.s3_sync_exclude
set -x
${command}
実行結果
% ./sync_dryrun.sh
+ aws s3 sync /path/to/dir/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude sync_dryrun.sh --exclude Gemfile --exclude Gemfile.lock --exclude .git/HEAD .git/branches .git/config .git/description .git/hooks .git/info .git/objects .git/refs --exclude .gitignore
Unknown options: .git/branches, .git/config, .git/description, .git/hooks, .git/info, .git/objects, .git/refs
zsh: exit 255 ./sync_dryrun.sh
*(アスタリスク)が展開されて大変なことになった…
*(アスタリスク)が展開されないようにダブルクォートで囲ってみる
sync_dryrun.sh
#!/bin/bash
dir=$(cd $(dirname ${0}); pwd)
file=${0##*/}
command="aws s3 sync ${dir}/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude ${file}"
while read exclude
do
command="${command} --exclude \"${exclude}\""
done < ${dir}/.s3_sync_exclude
set -x
${command}
実行結果
% ./sync_dryrun.sh
+ aws s3 sync /path/to/dir/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude sync_dryrun.sh --exclude '"Gemfile"' --exclude '"Gemfile.lock"' --exclude '".git/*"' --exclude '".gitignore"'
(dryrun) upload: .git/HEAD to s3://bucket_name/.git/HEAD
(dryrun) upload: .git/config to s3://bucket_name/.git/config
(dryrun) upload: .git/description to s3://bucket_name/.git/description
(dryrun) upload: .git/hooks/applypatch-msg.sample to s3://bucket_name/.git/hooks/applypatch-msg.sample
(dryrun) upload: .git/hooks/post-update.sample to s3://bucket_name/.git/hooks/post-update.sample
(dryrun) upload: .git/hooks/commit-msg.sample to s3://bucket_name/.git/hooks/commit-msg.sample
(dryrun) upload: .git/hooks/pre-commit.sample to s3://bucket_name/.git/hooks/pre-commit.sample
(dryrun) upload: .git/hooks/pre-applypatch.sample to s3://bucket_name/.git/hooks/pre-applypatch.sample
(dryrun) upload: .git/hooks/pre-push.sample to s3://bucket_name/.git/hooks/pre-push.sample
(dryrun) upload: .git/hooks/pre-rebase.sample to s3://bucket_name/.git/hooks/pre-rebase.sample
(dryrun) upload: .git/hooks/prepare-commit-msg.sample to s3://bucket_name/.git/hooks/prepare-commit-msg.sample
(dryrun) upload: .git/hooks/update.sample to s3://bucket_name/.git/hooks/update.sample
(dryrun) upload: .git/info/exclude to s3://bucket_name/.git/info/exclude
(dryrun) upload: ./Gemfile to s3://bucket_name/Gemfile
(dryrun) upload: ./Gemfile.lock to s3://bucket_name/Gemfile.lock
""で囲った文字列がさらに''で囲まれている
引数として""を含んだ文字列が渡されてexcludeしてもらえない
シングルクォートで囲んでみる
sync_dryrun.sh
#!/bin/bash
dir=$(cd $(dirname ${0}); pwd)
file=${0##*/}
command="aws s3 sync ${dir}/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude ${file}"
while read exclude
do
command="${command} --exclude '${exclude}'"
done < ${dir}/.s3_sync_exclude
set -x
${command}
実行結果
% ./sync_dryrun.sh
+ aws s3 sync /path/to/dir/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude sync_dryrun.sh --exclude ''\''Gemfile'\''' --exclude ''\''Gemfile.lock'\''' --exclude ''\''.git/*'\''' --exclude ''\''.gitignore'\'''
(dryrun) upload: .git/HEAD to s3://bucket_name/.git/HEAD
(dryrun) upload: .git/config to s3://bucket_name/.git/config
(dryrun) upload: .git/description to s3://bucket_name/.git/description
(dryrun) upload: .git/hooks/applypatch-msg.sample to s3://bucket_name/.git/hooks/applypatch-msg.sample
(dryrun) upload: .git/hooks/post-update.sample to s3://bucket_name/.git/hooks/post-update.sample
(dryrun) upload: .git/hooks/commit-msg.sample to s3://bucket_name/.git/hooks/commit-msg.sample
(dryrun) upload: .git/hooks/pre-commit.sample to s3://bucket_name/.git/hooks/pre-commit.sample
(dryrun) upload: .git/hooks/pre-applypatch.sample to s3://bucket_name/.git/hooks/pre-applypatch.sample
(dryrun) upload: .git/hooks/pre-push.sample to s3://bucket_name/.git/hooks/pre-push.sample
(dryrun) upload: .git/hooks/pre-rebase.sample to s3://bucket_name/.git/hooks/pre-rebase.sample
(dryrun) upload: .git/hooks/prepare-commit-msg.sample to s3://bucket_name/.git/hooks/prepare-commit-msg.sample
(dryrun) upload: .git/hooks/update.sample to s3://bucket_name/.git/hooks/update.sample
(dryrun) upload: .git/info/exclude to s3://bucket_name/.git/info/exclude
(dryrun) upload: ./Gemfile to s3://bucket_name/Gemfile
(dryrun) upload: ./Gemfile.lock to s3://bucket_name/Gemfile.lock
なんだかわからないけど'が大量に追加された。excludeされるはずもなく。。。
ダブルクォートに戻してからevalしてみる (解決編)
sync_dryrun.sh
#!/bin/bash
dir=$(cd $(dirname ${0}); pwd)
file=${0##*/}
command="aws s3 sync ${dir}/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude ${file}"
while read exclude
do
command="${command} --exclude \"${exclude}\""
done < ${dir}/.s3_sync_exclude
set -x
eval "${command}"
実行結果
% ./sync_dryrun.sh
+ eval 'aws s3 sync /path/to/dir/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude sync_dryrun.sh --exclude "Gemfile" --exclude "Gemfile.lock" --exclude ".git/*" --exclude ".gitignore"'
++ aws s3 sync /path/to/dir/ s3://bucket_name/ --delete --dryrun --exclude .s3_sync_exclude --exclude sync_dryrun.sh --exclude Gemfile --exclude Gemfile.lock --exclude '.git/*' --exclude .gitignore
""がなくなったり、*が含まれる文字列が''で囲まれたりしてるけど、正しい動きをしてくれた。