Go to Qiita Advent Calendar 2024 Top
LoginSignup
8
12

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

@nipe0324inファインディ株式会社

ActiveStorageでS3 Direct UploadしてS3に残ったゴミファイルを削除する

Posted at

確認バージョン

  • Rails 5.2.1
  • ActiveStorage 5.2.1

コマンド

こちらのコマンドをバッチなどで定期的に実行することで、S3 Direct UploadでS3上に残ってモデルからリンクされてないファイルを削除することができます。

# バックグラウンドでジョブが走って削除する
ActiveStorage::Blob.unattached.find_each(&:purge_later)

# ジョブが走って削除する
ActiveStorage::Blob.unattached.find_each(&:purge)

説明

unattachedスコープでは、Attachmentと紐づいてないBlobを取得します。ソース

class ActiveStorage::Blob < ActiveRecord::Base
  ...

  scope :unattached, -> { left_joins(:attachments).where(ActiveStorage::Attachment.table_name => { blob_id: nil }) }

  ...
end

S3 DirectUploadのときの流れとして、署名つきURL(Pre Signed URL)を取得する時に、Blobレコードを作成しており、モデルの保存時にバリデーションエラーなどが発生した場合に、Attachmentレコードに紐づいていないBlobレコードとS3上の画像が残ってしまいます。それを削除するコマンドが上記になります。

ログからより詳細

まずは簡単な設定です。

  • モデル
# app/models/message.rb
class Message < ApplicationRecord
  has_one_attached :attachment
end
  • コントローラー

# app/controllers/messages_controller.rb
class MessagesController < ApplicationController
  def create
    @message = Message.new(message_params)
    if @message.save
      redirect_to @message, notice: 'Message was successfully created.'
    else
      render :new
    end
  end

  private

    def message_params
      params.require(:message).permit(:comment, :attachment)
    end
end
  • ビュー
// app/views/messages/_form.html.erb
<%= form_with(model: message, local: true) do |form| %>
  <div class="field">
    <%= form.label :comment %>
    <%= form.text_area :comment %>
  </div>

  <div class="field">
    <%= form.label :attachment %>
    <%= form.file_field :attachment, direct_upload: true %>
  </div>

  <div class="actions">
    <%= form.submit %>
  </div>
<% end %>
  • javascript
// app/assets/javascripts/application.js
//
//= require rails-ujs
//= require activestorage
//= require turbolinks
//= require_tree .
  • フォームでSubmitボタンを押すと、JavascriptのActiveStorageから/rails/active_storage/direct_uploadsにアクセスします。このときに、ファイルの情報（ファイル名、コンテンツタイプなど）をサーバーに送り、Blobレコードを作成しています。
  • そして、BlobレコードからAWSのSDKを使い、Pre-signed URLを作成しJSに返しています。

Started POST "/rails/active_storage/direct_uploads" for 127.0.0.1 at 2018-08-10 02:42:21 +0900
Processing by ActiveStorage::DirectUploadsController#create as JSON
  Parameters: {"blob"=>{"filename"=>"image1.png", "content_type"=>"image/png", "byte_size"=>4598, "checksum"=>"t+FjbdAtkteQMlji0IEWVw=="}, "direct_upload"=>{"blob"=>{"filename"=>"image1.png", "content_type"=>"image/png", "byte_size"=>4598, "checksum"=>"t+FjbdAtkteQMlji0IEWVw=="}}}
   (0.1ms)  begin transaction
  ↳ /Users/hoge/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/activerecord-5.2.1/lib/active_record/log_subscriber.rb:98
  ActiveStorage::Blob Create (0.9ms)  INSERT INTO "active_storage_blobs" ("key", "filename", "content_type", "byte_size", "checksum", "created_at") VALUES (?, ?, ?, ?, ?, ?)  [["key", "c9kq2Tsmv1CwnPLKSxhZffwL"], ["filename", "image1.png"], ["content_type", "image/png"], ["byte_size", 4598], ["checksum", "t+FjbdAtkteQMlji0IEWVw=="], ["created_at", "2018-08-09 17:42:21.512543"]]
  ↳ /Users/hoge/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/activerecord-5.2.1/lib/active_record/log_subscriber.rb:98
   (1.7ms)  commit transaction
  ↳ /Users/hoge/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/activerecord-5.2.1/lib/active_record/log_subscriber.rb:98
  S3 Storage (1.6ms) Generated URL for file at key: c9kq2Tsmv1CwnPLKSxhZffwL (https://ytest-direct-upload.s3.amazonaws.com/c9kq2Tsmv1CwnPLKSxhZffwL?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIVSEX4F5TBW5I57A%2F20180809%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20180809T174221Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=0feec911ea4964721ac99be72daf22e454380a7b890b01afdf232a2a8161e700)
Completed 200 OK in 56ms (Views: 0.6ms | ActiveRecord: 3.3ms)

  • JSはPre-signed URLを取得できたので、それを使って、S3にダイレクトアップロードします。
    ダイレクトアップロードが成功したら、Railsのフォームのアクションを実行します。（ここでは PATCH /messages/3）。リクエストにBlobレコードを取得できる値(ここではattachemntの値)があるので、MessageモデルとBlobレコードを紐づけるAttachmentレコードが作成されます。

  • ここでMessageの保存の時にバリデーションエラーになったりすると、Attachmentレコードは作成されず、Blobレコードがひとりぼっちになってしまいます。
    そのため、最初のコマンドが必要になります。

Started POST "/messages" for 127.0.0.1 at 2018-08-10 02:42:22 +0900
Processing by MessagesController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"Fd++COckHOG+4EFZihATQv2aszWfibl9UClEANNcvrMpCOrUNRCgZ4oo/ZEwGWWkbNyrWoK/vo5HatzDKVNx+g==", "message"=>{"comment"=>"commnet", "attachment"=>"eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBKQT09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--99eddb1c71861e2b3f5749326a80960f04961760"}, "commit"=>"Create Message"}
  User Load (0.3ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = ? ORDER BY "users"."id" ASC LIMIT ?  [["id", 1], ["LIMIT", 1]]
  ↳ /Users/hoge/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/activerecord-5.2.1/lib/active_record/log_subscriber.rb:98
  ActiveStorage::Blob Load (0.1ms)  SELECT  "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = ? LIMIT ?  [["id", 31], ["LIMIT", 1]]
  ↳ app/controllers/messages_controller.rb:28
   (0.0ms)  begin transaction
  ↳ app/controllers/messages_controller.rb:28
   (0.1ms)  commit transaction
  ↳ app/controllers/messages_controller.rb:28
   (0.1ms)  begin transaction
  ↳ app/controllers/messages_controller.rb:31
  Message Create (0.4ms)  INSERT INTO "messages" ("user_id", "comment", "created_at", "updated_at") VALUES (?, ?, ?, ?)  [["user_id", 1], ["comment", "commnet"], ["created_at", "2018-08-09 17:42:22.720222"], ["updated_at", "2018-08-09 17:42:22.720222"]]
  ↳ app/controllers/messages_controller.rb:31
  ActiveStorage::Attachment Create (0.2ms)  INSERT INTO "active_storage_attachments" ("name", "record_type", "record_id", "blob_id", "created_at") VALUES (?, ?, ?, ?, ?)  [["name", "attachment"], ["record_type", "Message"], ["record_id", 4], ["blob_id", 31], ["created_at", "2018-08-09 17:42:22.721708"]]
  ↳ app/controllers/messages_controller.rb:31
  Message Update (0.1ms)  UPDATE "messages" SET "updated_at" = ? WHERE "messages"."id" = ?  [["updated_at", "2018-08-09 17:42:22.722738"], ["id", 4]]
  ↳ app/controllers/messages_controller.rb:31
   (1.0ms)  commit transaction
  ↳ app/controllers/messages_controller.rb:31
   (0.1ms)  begin transaction
  ↳ app/controllers/messages_controller.rb:31
  ActiveStorage::Blob Update (0.4ms)  UPDATE "active_storage_blobs" SET "metadata" = ? WHERE "active_storage_blobs"."id" = ?  [["metadata", "{\"identified\":true}"], ["id", 31]]
  ↳ app/controllers/messages_controller.rb:31
   (2.5ms)  commit transaction
  ↳ app/controllers/messages_controller.rb:31
[ActiveJob] Enqueued ActiveStorage::AnalyzeJob (Job ID: 118ebb60-f878-4c69-af80-afa47e65285e) to Async(default) with arguments: #<GlobalID:0x00007ff768a6bbe0 @uri=#<URI::GID gid://active-storage-sample/ActiveStorage::Blob/31>>
  ActiveStorage::Blob Load (0.4ms)  SELECT  "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = ? LIMIT ?  [["id", 31], ["LIMIT", 1]]
Redirected to http://localhost:3000/messages/4
Completed 302 Found in 886ms (ActiveRecord: 5.7ms)
8
12
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
8
12

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?