Skip to main content

Shrine 3.0.0

This guide covers all the changes in the 3.0.0 version of Shrine. If you're currently using Shrine 2.x, see Upgrading to Shrine 3.x for instructions on how to upgrade.

Major features​

Derivatives​

The new derivatives plugin has been added for storing additional processed files alongside the main file.

Shrine.plugin :derivatives
class ImageUploader < Shrine
  Attacher.derivatives_processor do |original|
    magick = ImageProcessing::MiniMagick.source(original) 

    {
      large:  magick.resize_to_limit!(800, 800),
      medium: magick.resize_to_limit!(500, 500),
      small:  magick.resize_to_limit!(300, 300),
    }
  end
end
photo = Photo.new(photo_params)
photo.image_derivatives! # creates derivatives
photo.save

This is a rewrite of the versions plugin, bringing numerous improvements:

  • processed files are separated from the main file

    photo.image_data #=>
# {
#   "id": "original.jpg",
#   "storage": "store",
#   "metadata": { ... },
#   "derivatives": {
#     "large": { "id": "large.jpg", "storage": "store", "metadata": { ... } },
#     "medium": { "id": "medium.jpg", "storage": "store", "metadata": { ... } },
#     "small": { "id": "small.jpg", "storage": "store", "metadata": { ... } }
#   }
# }

photo.image             #=> #<Shrine::UploadedFile id="original.jpg" ...>
photo.image_derivatives #=>
# {
#   large: #<Shrine::UploadedFile id="large.jpg" ...>,
#   medium: #<Shrine::UploadedFile id="medium.jpg" ...>,
#   small: #<Shrine::UploadedFile id="small.jpg" ...>,
# }

  • processing is decoupled from promotion

    photo = Photo.create(image: file) # promote original file to permanent storage
photo.image_derivatives!          # generate derivatives after promotion
photo.save                        # save derivatives data

  • ability to add or remove processed files at any point

    class ImageUploader < Shrine
  Attacher.derivatives_processor :thumbnails do |original|
    # ...
  end

  Attacher.derivatives_processor :crop do |original, left:, top:, width:, height:|
    vips = ImageProcessing::Vips.source(original)

    { cropped: vips.crop!(left, top, width, height) }
  end
end
    photo.image_derivatives!(:thumbnails)
photo.image_derivatives #=> { large: ..., medium: ..., small: ... }
photo.save

# ... sometime later ...

photo.image_derivatives!(:crop, left: 0, top: 0, width: 300, height: 300)
photo.image_derivatives #=> { large: ..., medium: ..., small: ..., cropped: ... }
photo.save

  • possibility of uploading processed files to different storage

    class ImageUploader < Shrine
  # specify storage for all derivatives
  Attacher.derivatives_storage :other_store

  # or specify storage per derivative
  Attacher.derivatives_storage { |derivative| :other_store }
end
    photo = Photo.create(image: file)
photo.image.storage_key #=> :store

photo.image_derivatives!
photo.image_derivatives[:large].storage_key #=> :other_store

Attacher redesign​

The Shrine::Attacher class has been rewritten and can now be used without models:

attacher = Shrine::Attacher.new
attacher.attach(file)
attacher.file #=> #<Shrine::UploadedFile>

The Attacher#data, Attacher#load_data, and Attacher.from_data methods have been added for dumping and loading the attached file:

# dump attached file into a serializable Hash
data = attacher.data #=> { "id" => "abc123.jpg", "storage" => "store", "metadata" => { ... } }
# initialize attacher from attached file data...
attacher = Shrine::Attacher.from_data(data)
attacher.file #=> #<Shrine::UploadedFile id="abc123.jpg" storage=:store metadata={...}>

# ...or load attached file into an existing attacher
attacher = Shrine::Attacher.new
attacher.load_data(data)
attacher.file #=> #<Shrine::UploadedFile>

Several more methods have been added:

  • Attacher#attach – attaches the file directly to permanent storage
  • Attacher#attach_cached – extracted from Attacher#assign
  • Attacher#upload – calls Shrine#upload, passing :record and :name context
  • Attacher#file – alias for Attacher#get
  • Attacher#cache_key – returns temporary storage key (:cache by default)
  • Attacher#store_key – returns permanent storage key (:store by default)

Column​

The new column plugin adds the ability to serialize attached file data, in format suitable for writing into a database column.

Shrine.plugin :column
# dump attached file data into a JSON string
data = attacher.column_data #=> '{"id":"abc123.jpg","storage":"store","metadata":{...}}'
# initialize attacher from attached file data...
attacher = Shrine::Attacher.from_column(data)
attacher.file #=> #<Shrine::UploadedFile id="abc123.jpg" storage=:store metadata={...}>

# ...or load attached file into an existing attacher
attacher = Shrine::Attacher.new
attacher.load_column(data)
attacher.file #=> #<Shrine::UploadedFile>

Entity​

The new entity plugin adds support for immutable structs, which are commonly used with ROM, Hanami and dry-rb.

Shrine.plugin :entity
class Photo < Hanami::Entity
  include Shrine::Attachment(:image)
end
photo = Photo.new(image_data: '{"id":"abc123.jpg","storage":"store","metadata":{...}}')
photo.image #=> #<Shrine::UploadedFile id="abc123.jpg" storage=:store ...>

Model​

The new model plugin adds support for mutable structs, which is used by activerecord and sequel plugins.

Shrine.plugin :model
class Photo < Struct.new(:image_data)
  include Shrine::Attachment(:image)
end
photo = Photo.new
photo.image = file
photo.image #=> #<Shrine::UploadedFile id="abc123.jpg" storage=:cache ...>
photo.image_data #=> #=> '{"id":"abc123.jpg", "storage":"cache", "metadata":{...}}'

Backgrounding rewrite​

  • The backgrounding plugin has been rewritten for more flexibility and simplicity. The new usage is much more explicit:
Shrine.plugin :backgrounding
Shrine::Attacher.promote_block do
  PromoteJob.perform_async(self.class.name, record.class.name, record.id, name, file_data)
end
Shrine::Attacher.destroy_block do
  DestroyJob.perform_async(self.class.name, data)
end
class PromoteJob
  include Sidekiq::Worker

  def perform(attacher_class, record_class, record.id, name, file_data)
    attacher_class = Object.const_get(attacher_class)
    record         = Object.const_get(record_class).find(record_id) # if using Active Record

    attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
    attacher.atomic_promote
  rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
    # attachment has changed or the record has been deleted, nothing to do
  end
end
class DestroyJob
  include Sidekiq::Worker

  def perform(attacher_class, data)
    attacher_class = Object.const_get(attacher_class)

    attacher = attacher_class.from_data(data)
    attacher.destroy
  end
end

There are several main differences compared to the old implementation:

  • we are in charge of passing the record to the background job
  • we can access the attacher before promotion
  • we can react to errors that caused promotion to abort

We can now also register backgrounding hooks on an attacher instance, allowing us to pass additional parameters to the background job:

photo = Photo.new(photo_params)

photo.image_attacher.promote_block do |attacher|
  PromoteJob.perform_async(
    attacher.class.name,
    attacher.record.class.name,
    attacher.record.id,
    attacher.name,
    attacher.file_data,
    current_user.id, # <== parameters from the controller
  )
end

photo.save # will call our instance-level backgrounding hook

Persistence interface​

The persistence plugins (activerecord, sequel) now implement a unified persistence interface:

MethodDescription
Attacher#persistpersists attachment data
Attacher#atomic_persistpersists attachment data if attachment hasn’t changed
Attacher#atomic_promotepromotes cached file and atomically persists changes

The "atomic" methods use the new atomic_helpers plugin, and are useful for background jobs. For example, this is how we'd use them to implement metadata extraction in the background in a concurrency-safe way:

MetadataJob.perform_async(
  attacher.class.name,
  attacher.record.class.name,
  attacher.record.id,
  attacher.name,
  attacher.file_data,
)
class MetadataJob
  include Sidekiq::Worker

  def perform(attacher_class, record_class, record_id, name, file_data)
    attacher_class = Object.const_get(attacher_class)
    record         = Object.const_get(record_class).find(record_id) # if using Active Record

    attacher = attacher_class.retrieve(model: record, name: name, file: file_data)
    attacher.refresh_metadata! # extract metadata
    attacher.atomic_persist    # persist if attachment hasn't changed
  rescue Shrine::AttachmentChanged, ActiveRecord::RecordNotFound
    # attachment has changed or record has been deleted, nothing to do
  end
end

Other new plugins​

  • The new mirroring plugin has been added for replicating uploads and deletes to other storages.

    Shrine.storages = { cache: ..., store: ..., backup: ... }

Shrine.plugin :mirroring, mirror: { store: :backup }
    file = Shrine.upload(io, :store) # uploads to :store and :backup
file.delete                      # deletes from :store and :backup

  • The new multi_cache plugin has been added for allowing an attacher to accept files from additional temporary storages.

    Shrine.storages = { cache: ..., cache_one: ..., cache_two: ..., store: ... }

Shrine.plugin :multi_cache, additional_cache: [:cache_one, :cache_two]
    photo.image = { "id" => "...", "storage" => "cache", "metadata" => { ... } }
photo.image.storage_key #=> :cache
# or
photo.image = { "id" => "...", "storage" => "cache_one", "metadata" => { ... } }
photo.image.storage_key #=> :cache_one
# or
photo.image = { "id" => "...", "storage" => "cache_two", "metadata" => { ... } }
photo.image.storage_key #=> :cache_two

  • The new form_assign plugin has been added for assigning files directly from form params.

    Shrine.plugin :form_assign
    attacher = photo.image_attacher
attacher.form_assign({ "image" => file, "title" => "...", "description" => "..." })
attacher.file #=> #<Shrine::UploadedFile id="..." storage=:cache ...>

Other features​

  • Model file assignment can now be configured to upload directly to permanent storage.

    Shrine.plugin :model, cache: false
    photo.image = file
photo.image.storage_key #=> :store (permanent storage)

  • New Shrine.download_response method has been added to the download_endpoint plugin for generating file response from the controller.

    Rails.application.routes.draw do
  get "/attachments" => "files#download"
end
    class FilesController < ApplicationController
  def download
    # ... we can now perform things like authentication here ...
    set_rack_response Shrine.download_response(env)
  end

  private

  def set_rack_response((status, headers, body))
    self.status = status
    self.headers.merge!(headers)
    self.response_body = body
  end
end

  • The Attacher#refresh_metadata! method has been added to refresh_metadata plugin. It refreshes metadata and writes new attached file data back into the data attribute.

    attacher.file.refresh_metadata!
attacher.write
# can now be shortened to
attacher.refresh_metadata!

  • Including a Shrine::Attachment module now defines a .<name>_attacher class method on the target class.

    class Photo
  include ImageUploader::Attachment(:image)
end
    Photo.image_attacher #=> #<ImageUploader::Attacher ...>

  • The attachment data serializer is now configurable (by default JSON standard library is used):

    require "oj" # https://github.com/ohler55/oj

Shrine.plugin :column, serializer: Oj

  • It's now possible to pass options to the validate block via the :validate option:

    attacher.assign(file, validate: { foo: "bar" })
    class MyUploader < Shrine
  Attacher.validate do |**options|
    options #=> { foo: "bar" }
  end
end

  • Validation can now be skipped on assignment by passing validate: false.

    attacher.attach(file, validate: false) # skip validation

  • Closing the uploaded file can now be prevented by passing close: false to Shrine#upload.

  • Uploaded file can now be automatically deleted by passing delete: true to Shrine#upload.

  • New Attacher#file! method has been added for retrieving the attached file and raising and exception if it doesn't exist.

  • New Derivation#opened method has been added for retrieving an opened derivative in derivation_endpoint plugin.

  • New Storage#delete_prefixed method has been added for deleting all files in specified directory.

    storage.delete_prefixed("some_directory/")

Performance improvements​

  • The attached file is now parsed and loaded from record column only once, which can greatly improve performance if the same attached file is being accessed multiple times.

    photo = Photo.find(photo_id)
photo.image # parses and loads attached file
photo.image # returns memoized attached file

  • The S3#open method doesn't perform a #head_obect request anymore.

  • The derivation_endpoint plugin doesn't perform a Storage#exists? call anymore when :upload is enabled.

  • The download_endpoint plugin doesn't perform a Storage#exists? call anymore.

Other Improvements​

Core improvements​

  • Shrine now works again with MRI 2.3.

  • The memory storage from the shrine-memory gem has been merged into core.

    # Gemfile
gem "shrine-memory" # this can be removed

  • The Attacher#assign method now accepts cached file data as a Hash.

    photo.image = { "id" => "...", "storage" => "cache", "metadata" => { ... } }

  • An UploadedFile object can now be initialized with symbol keys.

    Shrine.uploaded_file(id: "...", storage: :store, metadata: { ... })

  • The temporary storage doesn't need to be defined anymore if it's not used.

  • Any changes to Shrine.storages will now be applied to existing Shrine and Attacher instances.

  • When copying the S3 object to another location, any specified upload options will now be applied.

  • Deprecation of passing unknown options to FileSystem#open has been reverted. This allows users to continue using FileSystem storage in tests as a mock storage.

  • The Shrine#upload method now infers file extension from filename metadata, making possible to use filename to specify file extension.

    file = uploader.upload(StringIO.new("some text"), metadata: { "filename" => "file.txt" })
file.id #=> "2a2467ee6acbc5cb.txt"

  • The Shrine.opts hash is now deep-copied on subclassing. This allows plugins to freely mutate hashes and arrays in Shrine.opts, knowing they won't be shared across subclasses.

  • The down dependency has been updated to ~> 5.0.

  • The Shrine::Attachment[] method has been added as an alternative syntax for creating attachment modules.

    class Photo
  include ImageUploader::Attachment[:image]
end

Plugin improvements​

  • The activerecord plugin now works with Active Record 3.

  • Callback code from activerecord and sequel plugin has been moved into attacher methods, allowing the user to override them.

    • Attacher#(activerecord|sequel)_before_save
    • Attacher#(activerecord|sequel)_after_save
    • Attacher#(activerecord|sequel)_after_destroy

  • The url_options plugin now allows you to override URL options by deleting them.

    uploaded_file.url(response_content_disposition: "attachment")
    plugin :url_options, store: -> (io, options) {
  disposition = options.delete(:response_content_disposition, "inline")

  {
    response_content_disposition: ContentDisposition.format(
      disposition: disposition,
      filename:    io.original_filename,
    )
  }
}

  • The :upload_options hash passed to the uploader is now merged with any options defined with the upload_options plugin.

  • The default_storage now evaluates the storage block in context of the Attacher instance.

  • New Attacher.default_cache and Attacher.default_store methods have been added to the default_storage plugin for declaratively setting default storage.

    Attacher.default_cache { ... }
Attacher.default_store { ... }

  • The derivation_endpoint plugin now handles string derivation names.

  • The Derivation#upload method from derivation_endpoint plugin now accepts additional uploader options

  • The Derivation#upload method from derivation_endpoint plugin now accepts any IO-like object.

  • The derivation_endpoint plugin doesn't re-open File objects returned in derivation block anymore.

  • The instrumentation plugin now instruments UploadedFile#open calls as a new open.shrine event. UploadedFile#download is still instrumented as download.shrine.

  • The upload.shrine event now has :metadata on the top level in instrumentation plugin.

  • The width & height validators in store_dimensions plugin don't require UploadedFile#width and UploadedFile#height methods to be defined anymore, only that the corresponding metadata exists.

  • Any options passed to Attacher#attach_cached are now forwarded to metadata extraction when restore_cached_data plugin is loaded.

  • The infer_extension plugin now works correctly with pretty_location plugin when pretty_location was loaded after infer_extension.

  • The pretty_location plugin now accepts :class_underscore option for underscoring class names.

    plugin :pretty_location
# "blogpost/aa357797-5845-451b-8662-08eecdc9f762/image/493g82jf23.jpg"

plugin :pretty_location, class_underscore: :true
# "blog_post/aa357797-5845-451b-8662-08eecdc9f762/image/493g82jf23.jpg"

  • You can now load multiple persistence plugins simulatenously, and the correct one will be activated during persistence.

Backwards compatibility​

Plugin deprecation and removal​

  • The backgrounding plugin has been rewritten and has a new API. While the new API works in a similar way, no backwards compatibility has been kept with the previous API.

  • The versions, processing, recache, and delete_raw plugins have been deprecated in favor of the new derivatives plugin.

  • The module_include plugin has been deprecated over overriding core classes directly.

  • The hooks, parallelize, parsed_json, and delete_promoted plugins have been removed.

  • The deprecated copy, backup, multi_delete, moving, logging, direct_upload background_helpers, and migration_helpers plugins have been removed.

  • The default_url_options plugin has been renamed to url_options.

Attacher API​

  • The Attacher.new method now only accepts a hash of options, use Attacher.from_model for initializing from a model.

  • If you're changing the attachment data column directly, you'll now need to call Attacher#reload to make the attacher reflect those changes.

  • The Attacher#promote method now only saves the promoted file in memory, it doesn't persist the changes.

  • The Attacher#_set and Attacher#set methods have been renamed to Attacher#set and Attacher#changed.

  • The Attacher#cache!, Attacher#store!, and Attacher#delete! methods have been removed.

  • The Attacher#_promote and Attacher#_delete methods have been removed.

  • The Attacher#swap, Attacher#update, Attacher#read, Attacher#write, Attacher#convert_to_data, Attacher#convert_before_write, and Attache#convert_after_read methods have been removed.

  • The Attacher.validate, Attacher#validate and Attacher#errors methods have been extracted into the new validation plugin.

  • The Attacher#data_attribute method has been renamed to Attacher#attribute.

  • The Attacher#replace method has been renamed to Attacher#destroy_previous.

  • The Attacher#assign method now raises an exception when non-cached uploaded file is assigned.

  • The Attacher#attached? method now returns whether a file is attached, regardless of whether it was changed or not.

Attachment API​

  • The Shrine::Attachment module doesn't define any instance methods by itself anymore. This has been moved into entity and model plugins.

Uploader API​

  • The :phase option has been removed.

  • The Shrine#process and Shrine#processed methods have been removed.

  • The Shrine#store, Shrine#_store, Shrine#put, and Shrine#copy methods have been removed.

  • The Shrine#delete, Shrine#_delete, and Shrine#remove methods have been removed.

  • The Shrine#uploaded? method has been removed.

  • The Shrine.uploaded_file method doesn't yield files anymore by default.

  • The options for Shrine#upload and Shrine#extract_metadata are now required to have symbol keys.

  • The Shrine.uploaded_file method now raises ArgumentError on invalid arguments.

  • The Shrine#upload method doesn't rescue exceptions that happen in IO#close anymore.

  • The deprecated Shrine::IO_METHODS constant has been removed.

Uploaded File API​

  • The UploadedFile method now extracts data from the given hash on initialization, it doesn't store the whole hash anymore. This means the following potential code won't work anymore:

    uploaded_file.id #=> "foo"
uploaded_file.data["id"] = "bar"
uploaded_file.id #=> "foo"

  • The UploadedFile#storage_key method now returns a Symbol instead of a String.

    # previous behaviour
uploaded_file.storage_key #=> "store"

# new behaviour
uploaded_file.storage_key #=> :store

  • The UploadedFile#== method now requires both uploaded file objects to be of the same class.

Storage API​

  • The Storage#open method is now required to accept additional options.

    # this won't work anymore
def open(id)
  # ...
end

# this is now required
def open(id, **options)
  # ...
end

  • The Storage#open method is now required to raise Shrine::FileNotFound exception when the file is missing.

S3 API​

  • The support for aws-sdk 2.x and aws-sdk-s3 < 1.14 has been removed.

  • S3#open now raises Shrine::FileNotFound exception when S3 object doesn't exist on the bucket.

  • The S3#upload method will now override any S3 object data when copying from another S3 object. If you were relying on S3 object data being inherited on copy, you will need to update your code.

  • The :host option in S3#initialize has been removed.

  • The :download option in S3#url has been removed.

  • Specifying :multipart_threshold as an integer is not supported anymore.

  • Non URI-escaped :content_disposition and :response_content_disposition values are not supported anymore.

  • The S3#presign method now returns a hash instead of an object for default POST method.

  • The deprecated S3#stream, S3#download, and S3#s3 methods have been removed.

  • The S3#open method doesn't include an :object in Down::ChunkedIO#data anymore.

FileSystem API​

  • FileSystem#open now raises Shrine::FileNotFound exception when file does not exist on the filesystem.

  • The deprecated :host option in FileSystem#initialize has been removed.

  • The deprecated :older_than option in FileSystem#clear! has been removed.

  • The deprecated FileSystem#download method has been removed.

  • The FileSystem#movable? and FileSystem#move methods have been made private.

  • The FileSystem#open method doesn't accept a block anymore.

Plugins API​

  • remote_url

    • A custom downloader is now required to raise Shrine::Plugins::RemoteUrl::DownloadError in order for the exception to be converted into a validation error. Down::NotFound and Down::TooLarge exceptions are converted by default. All other exceptions are propagated.

  • upload_endpoint

    • The deprecated Shrine::Plugins::UploadEndpoint::App constant has been removed.

    • The :request option holding the Rack::Request object isn't passed to the uploader anymore.

  • presign_endpoint

    • Storage#presign results that cannot coerce themselves into a Hash are not supported anymore.

    • The deprecated Shrine::Plugins::PresignEndpoint::App constant has been removed.

  • derivation_endpoint

    • The derivation block is now evaluated in context of a Shrine::Derivation instance.

    • The derivation block can now return only File and Tempfile objects.

    • The :download_errors option has been removed, as it's now obsolete.

    • The :include_uploaded_file option has been removed, as it's now obsolete.

    • The source UploadedFile is not passed to the derivation block anymore on download: false.

    • The Derivation#upload method now closes the uploaded file.

    • The derivation.upload instrumentation event payload now includes only :derivation key.

  • download_endpoint

    • Support for legacy /:storage/:id URLs has been dropped.

    • The :storages plugin option has been removed.

    • The Shrine::Plugins::DownloadEndpoint::App constant has been removed.

  • data_uri

    • The deprecated :filename plugin option has been removed.

    • The deprecated Shrine::Plugins::DataUri::DataFile constant has been removed.

  • rack_file

    • The deprecated Shrine::Plugins::RackFile::UploadedFile constant has been removed.

    • Passing a Rack uploaded file hash to Shrine#upload is not supported anymore.

  • cached_attachment_data

    • The #<name>_cached_data= model method has been removed.

    • The Attacher#read_cached method has been renamed to Attacher#cached_data.

  • default_url

    • Passing a block when loading the plugin is not supported anymore.

  • determine_mime_type

    • The deprecated :default analyzer alias has been removed.

    • The private Shrine#mime_type_analyzers method has been removed.

  • store_dimensions

    • Failing to extract dimensions now prints out a warning by default.

    • The private Shrine#extract_dimensions and Shrine#dimensions_analyzers methods have been removed.

  • infer_extension

    • The :mini_mime inferrer is now the default inferrer, which requires the mini_mime gem.

    • The private Shrine#infer_extension method has been removed.

  • validation_helpers

    • The width & height validators will now raise an exception if width or height metadata is missing.

    • Support for regexes has been dropped for MIME type and extension validators.

  • versions

    • The deprecated :version_names, Shrine.version_names and Shrine.version? have been removed from versions plugin.

  • keep_files

    • The plugin will now always prevent deletion of both replaced and destroyed attachments. The :replaced and :destroyed options don't have effect anymore.

  • dynamic_storage

    • The Shrine.dynamic_storages method has been removed.

  • instrumentation

    • The :location, :upload_options, and :metadata keys have been removed from :options in upload.shrine event payload.

    • The :metadata key has been removed from metadata.shrine event payload.

  • default_storage

    • Passing record & name arguments to the storage block has been deprecated over evaluating the block in context of the attacher instance.

  • sequel

    • The :callbacks plugin option has been renamed to :hooks.