class Shrine::Storage::S3

  1. lib/shrine/storage/s3.rb
Superclass: Object

Constants

COPY_OPTIONS = { tagging_directive: "REPLACE" }  
MAX_MULTIPART_PARTS = 10_000  
MIN_PART_SIZE = 5*1024*1024  
MULTIPART_THRESHOLD = { upload: 15*1024*1024, copy: 100*1024*1024 }  

Attributes

bucket [R]
client [R]
copy_options [R]
prefix [R]
public [R]
signer [R]
upload_options [R]

Public Class methods

new(bucket:, client: nil, prefix: nil, upload_options: {}, multipart_threshold: {}, max_multipart_parts: nil, signer: nil, public: nil, copy_options: COPY_OPTIONS, **)

Initializes a storage for uploading to S3. All options are forwarded to Aws::S3::Client#initialize, except the following:

:bucket

(Required). Name of the S3 bucket.

:client

By default an Aws::S3::Client instance is created internally from additional options, but you can use this option to provide your own client. This can be an Aws::S3::Client or an Aws::S3::Encryption::Client object.

:prefix

“Directory” inside the bucket to store files into.

:upload_options

Additional options that will be used for uploading files, they will be passed to Aws::S3::Object#put, Aws::S3::Object#copy_from and Aws::S3::Bucket#presigned_post.

:copy_options

Additional options that will be used for copying files, they will be passed to Aws::S3::Object#copy_from.

:multipart_threshold

If the input file is larger than the specified size, a parallelized multipart will be used for the upload/copy. Defaults to {upload: 15*1024*1024, copy: 100*1024*1024} (15MB for upload requests, 100MB for copy requests).

:max_multipart_parts

Limits the number of parts if parellized multipart upload/copy is used. Defaults to 10_000.

In addition to specifying the :bucket, you’ll also need to provide AWS credentials. The most common way is to provide them directly via :access_key_id, :secret_access_key, and :region options. But you can also use any other way of authentication specified in the AWS SDK documentation.

[show source] 
   # File lib/shrine/storage/s3.rb
71 def initialize(bucket:, client: nil, prefix: nil, upload_options: {}, multipart_threshold: {}, max_multipart_parts: nil, signer: nil, public: nil, copy_options: COPY_OPTIONS, **)
72   raise ArgumentError, "the :bucket option is nil" unless bucket
73 
74   @client = client || Aws::S3::Client.new(**)
75   @transfer_manager = Aws::S3::TransferManager.new(client: @client) if defined?(Aws::S3::TransferManager)
76   @bucket = Aws::S3::Bucket.new(name: bucket, client: @client)
77   @prefix = prefix
78   @upload_options = upload_options
79   @copy_options = copy_options
80   @multipart_threshold = MULTIPART_THRESHOLD.merge(multipart_threshold)
81   @max_multipart_parts = max_multipart_parts || MAX_MULTIPART_PARTS
82   @signer = signer
83   @public = public
84 end

Public Instance methods

clear!(&block)

If block is given, deletes all objects from the storage for which the block evaluates to true. Otherwise deletes all objects from the storage.

s3.clear!
# or
s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 }
[show source] 
    # File lib/shrine/storage/s3.rb
219 def clear!(&block)
220   objects_to_delete = bucket.objects(prefix: prefix)
221   objects_to_delete = objects_to_delete.lazy.select(&block) if block
222 
223   delete_objects(objects_to_delete)
224 end
delete(id)

Deletes the file from the storage.

[show source] 
    # File lib/shrine/storage/s3.rb
199 def delete(id)
200   object(id).delete
201 end
delete_prefixed(delete_prefix)

Deletes objects at keys starting with the specified prefix.

s3.delete_prefixed(“somekey/derivatives/”)

[show source] 
    # File lib/shrine/storage/s3.rb
206 def delete_prefixed(delete_prefix)
207   # We need to make sure to combine with storage prefix, and
208   # that it ends in '/' cause S3 can be squirrely about matching interior.
209   delete_prefix = delete_prefix.chomp("/") + "/"
210   bucket.objects(prefix: [*prefix, delete_prefix].join("/")).batch_delete!
211 end
exists?(id)

Returns true file exists on S3.

[show source] 
    # File lib/shrine/storage/s3.rb
128 def exists?(id)
129   object(id).exists?
130 end
object(id)

Returns an Aws::S3::Object for the given id.

[show source] 
    # File lib/shrine/storage/s3.rb
227 def object(id)
228   bucket.object(object_key(id))
229 end
open(id, rewindable: true, encoding: nil, **)

Returns a Down::ChunkedIO object that downloads S3 object content on-demand. By default, read content will be cached onto disk so that it can be rewinded, but if you don’t need that you can pass rewindable: false. A required character encoding can be passed in encoding; the default is Encoding::BINARY via Down::ChunkedIO.

Any additional options are forwarded to Aws::S3::Object#get.

[show source] 
    # File lib/shrine/storage/s3.rb
119 def open(id, rewindable: true, encoding: nil, **)
120   chunks, length = get(id, **)
121 
122   Down::ChunkedIO.new(chunks: chunks, rewindable: rewindable, size: length, encoding: encoding)
123 rescue Aws::S3::Errors::NoSuchKey
124   raise Shrine::FileNotFound, "file #{id.inspect} not found on storage"
125 end
presign(id, method: :post, **presign_options)

Returns URL, params, headers, and verb for direct uploads.

s3.presign("key") #=>
# {
#   url: "https://my-bucket.s3.amazonaws.com/...",
#   fields: { ... },  # blank for PUT presigns
#   headers: { ... }, # blank for POST presigns
#   method: "post",
# }

By default it calls Aws::S3::Object#presigned_post which generates data for a POST request, but you can also specify method: :put for PUT uploads which calls Aws::S3::Object#presigned_url.

s3.presign("key", method: :post) # for POST upload (default)
s3.presign("key", method: :put)  # for PUT upload

Any additional options are forwarded to the underlying AWS SDK method.

[show source] 
    # File lib/shrine/storage/s3.rb
189 def presign(id, method: :post, **presign_options)
190   options = {}
191   options[:acl] = "public-read" if public
192 
193   options.merge!(@upload_options, presign_options)
194 
195   send(:"presign_#{method}", id, options)
196 end
upload(io, id, shrine_metadata: {}, **upload_options)

If the file is an UploadedFile from S3, issues a COPY command, otherwise uploads the file. For files larger than :multipart_threshold a multipart upload/copy will be used for better performance and more resilient uploads.

It assigns the correct “Content-Type” taken from the MIME type, because by default S3 sets everything to “application/octet-stream”.

[show source] 
    # File lib/shrine/storage/s3.rb
 93 def upload(io, id, shrine_metadata: {}, **upload_options)
 94   content_type, filename = shrine_metadata.values_at("mime_type", "filename")
 95 
 96   options = {}
 97   options[:content_type] = content_type if content_type
 98   options[:content_disposition] = ContentDisposition.inline(filename) if filename
 99   options[:acl] = "public-read" if public
100 
101   options.merge!(@upload_options, upload_options)
102 
103   if copyable?(io)
104     copy(io, id, **options)
105   else
106     put(io, id, **options)
107   end
108 end
url(id, public: self.public, host: nil, **)

Returns the presigned URL to the file.

:host

This option replaces the host part of the returned URL, and is typically useful for setting CDN hosts (e.g. http://abc123.cloudfront.net)

:public

Returns the unsigned URL to the S3 object. This requires the S3 object to be public.

All other options are forwarded to Aws::S3::Object#presigned_url or Aws::S3::Object#public_url.

[show source] 
    # File lib/shrine/storage/s3.rb
148 def url(id, public: self.public, host: nil, **)
149   if public || signer
150     url = object(id).public_url(**)
151   else
152     url = object(id).presigned_url(:get, **)
153   end
154 
155   if host
156     uri = URI.parse(url)
157     uri.path = uri.path.match(/^\/#{bucket.name}/).post_match unless uri.host.include?(bucket.name)
158     url = URI.join(host, uri.request_uri[1..-1]).to_s
159   end
160 
161   if signer
162     url = signer.call(url, **)
163   end
164 
165   url
166 end