14
12

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

autofsでs3fsを利用する

Last updated at Posted at 2013-03-13

s3fsが便利なのでよく利用していますが、
大量のファイルをコピー/移動するとs3fsプロセスのメモリ使用量が増えてしまいます。
s3fsをremountしたら良いのですが、その度にコマンド叩くのは面倒なのでautofsで管理することにしました。

s3fsはcloudpackさんのs3fs-cloudpack を利用しています。

1. cookbook作成

$ knife cookbook create autofs
$ knife cookbook create s3fs-cloudpack

2. autofsのcookbook定義

autofs/recipe/default/default.rb
package "autofs"

template "/etc/auto.misc" do
  source "auto.misc.erb"
  owner "root"
  group "root"
  mode  "0644"
  notifies :restart, "service[autofs]", :immediately
end

service "autofs" do
  action :enable
end
autofs/template/default/auto.misc.erb
<%= node["s3fs_mountpoint"] %>  -fstype=fuse,allow_other,url=<%= node["s3fs_url"] %>  :s3fs#<%= node["s3fs_bucketname"] %>
autofs/attribute/default.rb
node.set["s3fs_mountpoint"] = "s3fs"
node.set["s3fs_url"] = "https://s3.amazonaws.com"
node.set["s3fs_bucketname"] = "hogebucket"

3. s3fs-cloudpackのcookbook定義

色々デフォルト。
scriptリソースは改良の余地あり。

s3fs-cloudpack/recipes/default.rb
s3fs_requirepkg = ["libcurl-devel","libxml2-devel","libgcrypt-devel","openssl-devel"]
s3fs_requirepkg.each do |p|
  package "#{p}" do
    action :install
  end
end

if ! File.exists?("/usr/local/bin/s3fs")
  template "/etc/ld.so.conf.d/local-lib.conf" do
    source "local-lib.conf.erb"
  end

  script "s3fs-install" do
    interpreter "bash"
    user "root"
    cwd "/tmp"
    code <<-EOF
      wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9.2/fuse-2.9.2.tar.gz
      tar zxf fuse-2.9.2.tar.gz
      cd fuse-2.9.2
      ./configure && make && make install
      export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
      git clone https://github.com/memorycraft/s3fs-cloudpack.git
      cd s3fs-cloudpack
      ldconfig
      ./configure && make && make install
    EOF
  end
end

template "/etc/passwd-s3fs" do
  source "passwd-s3fs.erb"
  owner "root"
  group "root"
  mode  "0600"
end

attributeでaccess_keyとsecret_keyを設定していますが、
databagを利用したほうがよいですね

s3fs-cloudpack/attributes/default.rb
node.set["access_key"] = "AAAAAAAAAAAAAAAA"
node.set["secret_key"] = "BBBBBBBBBBBBBBBBBBBBBBB"
s3fs-cloudpack/templates/default/local-lib.conf.erb
/usr/local/lib
s3fs-cloudpack/templates/default/passwd-s3fs.erb
<%= node.chef_environment["access_key"] %>:<%= node.chef_environment["secret_key"] %>

4. provision

一先ずVagrantで動作確認しました

$ vagrant provision
※参考:Vagrantfile
Vagrant::Config.run do |config|
  config.vm.box = "centos"
  config.vm.customize do |v|
        v.memory_size = 256
  end

  config.vm.provision :chef_solo do |chef|
  chef.cookbooks_path = ["<自分のcookbook-directory>"]
  chef.add_recipe "s3fs-cloudpack"
  chef.add_recipe "autofs"
  end

end

5. 確認

※s3fsマウント中
# ps aux|grep s3fs
root      9845  0.0  1.6 326188 10132 ?      Ssl  22:05   0:00 s3fs <bucketName> /misc/<mountPoint> -o rw,allow_other,url=https://s3.amazonaws.com,dev,suid
※デフォルトだと10分アイドルタイムが発生するとマウント解除される
# ps aux|grep s3fs
root      9882  0.0  0.1 103232   868 pts/0    S+   22:16   0:00 grep s3fs
※再アクセス
# df /misc/<mountPoint>
# ps aux|grep s3fs
root      9890  0.0  0.3 247296  2412 ?        Ssl  22:17   0:00 s3fs <bucketName> /misc/<mountPoint> -o rw,allow_other,url=https://s3.amazonaws.com,dev,suid

14
12
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
14
12

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?