— id: changing-derivatives
title: Managing Derivatives¶ ↑
This guide shows how to add, create, update, and remove derivatives for an app in production already handling file attachments, with zero downtime.
Let’s assume we have a Photo
model with an image
file attachment. The examples will be showing image thumbnails, but the advice applies to any kind of derivatives.
Shrine.plugin :derivatives Shrine.plugin :activerecord
class ImageUploader < Shrine # ... end
class Photo < ActiveRecord::Base include ImageUploader::Attachment(:image) end
Adding derivatives¶ ↑
Scenario: Your app is currently working only with original files, and you want to introduce derivatives.
You’ll first want to start creating the derivatives in production, without yet generating URLs for them (because existing attachments won’t yet have derivatives generated). Let’s assume you’re generating image thumbnails:
# Gemfile gem "image_processing", "~> 1.8"
require "image_processing/mini_magick" class ImageUploader < Shrine Attacher.derivatives do |original| magick = ImageProcessing::MiniMagick.source(original) # generate the thumbnails you want here { small: magick.resize_to_limit!(300, 300), medium: magick.resize_to_limit!(500, 500), large: magick.resize_to_limit!(800, 800), } end end
photo = Photo.new(photo_params) photo.image_derivatives! # generate derivatives photo.save
Once we’ve deployed this to production, we can run the following script to generate derivatives for all existing attachments in production. It fetches the records in batches, downloads attachments on permanent storage, creates derivatives, and persists the changes.
Photo.find_each do |photo| attacher = photo.image_attacher next unless attacher.stored? attacher.create_derivatives begin attacher.atomic_persist # persist changes if attachment has not changed in the meantime rescue Shrine::AttachmentChanged, # attachment has changed ActiveRecord::RecordNotFound # record has been deleted attacher.delete_derivatives # delete now orphaned derivatives end end
Now all attachments should have correctly generated derivatives. You can update the attachment URLs to use derivatives as needed.
Reprocessing all derivatives¶ ↑
Scenario: The processing logic has changed for all or most derivatives, and now you want to reprocess them for existing attachments.
Let’s assume we’ve made the following change and have deployed it to production:
Attacher.derivatives do |original| magick = ImageProcessing::MiniMagick.source(original) + .saver(quality: 85) { small: magick.resize_to_limit!(300, 300), medium: magick.resize_to_limit!(500, 500), large: magick.resize_to_limit!(800, 800), } end
We can now run the following script to reprocess derivatives for all existing records. It fetches the records in batches, downloads attachments on permanent storage, reprocesses new derivatives, persists the changes, and deletes old derivatives.
Photo.find_each do |photo| attacher = photo.image_attacher next unless attacher.stored? old_derivatives = attacher.derivatives attacher.set_derivatives({}) # clear derivatives attacher.create_derivatives # reprocess derivatives begin attacher.atomic_persist # persist changes if attachment has not changed in the meantime attacher.delete_derivatives(old_derivatives) # delete old derivatives rescue Shrine::AttachmentChanged, # attachment has changed ActiveRecord::RecordNotFound # record has been deleted attacher.delete_derivatives # delete now orphaned derivatives end end
Reprocessing certain derivatives¶ ↑
Scenario: The processing logic has changed for specific derivatives, and now you want to reprocess them for existing attachments.
Let’s assume we’ve made a following change and have deployed it to production:
Attacher.derivatives do |original| magick = ImageProcessing::MiniMagick.source(original) { small: magick.resize_to_limit!(300, 300), - medium: magick.resize_to_limit!(500, 500), + medium: magick.resize_to_limit!(600, 600), large: magick.resize_to_limit!(800, 800), } end
We can now run the following script to reprocess the derivative for all existing records. It fetches the records in batches, downloads attachments with derivatives, reprocesses the specific derivative, persists the change, and deletes old derivative.
Photo.find_each do |photo| attacher = photo.image_attacher next unless attacher.derivatives.key?(:medium) old_medium = attacher.derivatives[:medium] new_medium = attacher.file.download do |original| ImageProcessing::MiniMagick .source(original) .resize_to_limit!(600, 600) end attacher.add_derivative(:medium, new_medium) begin attacher.atomic_persist # persist changes if attachment has not changed in the meantime old_medium.delete # delete old derivative rescue Shrine::AttachmentChanged, # attachment has changed ActiveRecord::RecordNotFound # record has been deleted attacher.derivatives[:medium].delete # delete now orphaned derivative end end
Adding new derivatives¶ ↑
Scenario: A new derivative has been added to the processor, and now you want to add it to existing attachments.
Let’s assume we’ve made a following change and have deployed it to production:
Attacher.derivatives do |original| magick = ImageProcessing::MiniMagick.source(original) { + square: magick.resize_to_fill!(150, 150), small: magick.resize_to_limit!(300, 300), medium: magick.resize_to_limit!(600, 600), large: magick.resize_to_limit!(800, 800), } end
We can now run following script to add the new derivative for all existing records. It fetches the records in batches, downloads attachments on permanent storage, creates the new derivative, and persists the changes.
Photo.find_each do |photo| attacher = photo.image_attacher next unless attacher.stored? square = attacher.file.download do |original| ImageProcessing::MiniMagick .source(original) .resize_to_fill!(150, 150) end attacher.add_derivative(:square, square) begin attacher.atomic_persist # persist changes if attachment has not changed in the meantime rescue Shrine::AttachmentChanged, # attachment has changed ActiveRecord::RecordNotFound # record has been deleted attacher.derivatives[:square].delete # delete now orphaned derivative end end
Now all attachments should have the new derivative and you can start generating URLs for it.
Removing derivatives¶ ↑
Scenario: A derivative isn’t being used anymore, so we want to delete it for existing attachments.
Let’s assume we’ve made the following change and have deployed it to production:
Attacher.derivatives do |original| magick = ImageProcessing::MiniMagick.source(original) { - square: magick.resize_to_fill!(150, 150), small: magick.resize_to_limit!(300, 300), medium: magick.resize_to_limit!(600, 600), large: magick.resize_to_limit!(800, 800), } end
We can now run following script to remove the unused derivative for all existing record. It fetches the records in batches, removes and deletes the unused derivative, and persists the changes.
Photo.find_each do |photo| attacher = photo.image_attacher next unless attacher.derivatives.key?(:square) attacher.remove_derivative(:square, delete: true) begin attacher.atomic_persist # persist changes if attachment has not changed in the meantime rescue Shrine::AttachmentChanged, # attachment has changed ActiveRecord::RecordNotFound # record has been deleted end end
Backgrounding¶ ↑
For faster migration, we can also delay any of the operations above into a background job:
Photo.find_each do |photo| attacher = photo.image_attacher next unless attacher.stored? MakeChangeJob.perform_async( attacher.class.name, attacher.record.class.name, attacher.record.id, attacher.name, attacher.file_data, ) end
class MakeChangeJob include Sidekiq::Worker def perform(attacher_class, record_class, record_id, name, file_data) attacher_class = Object.const_get(attacher_class) record = Object.const_get(record_class).find(record_id) # if using Active Record attacher = attacher_class.retrieve(model: record, name: name, file: file_data) # ... make our change ... end end