Shrine

Shrine

  • Guides
  • Plugins
  • External
  • Discourse
  • GitHub
  • Wiki

›Migrating

Introduction

  • Advantages of Shrine
  • Getting Started

Base

  • The Design of Shrine
  • Retrieving Uploads
  • Using Attacher

Storage

  • File System
  • AWS S3
  • Memory

Features

  • Direct Uploads to S3
  • Extracting Metadata
  • File Processing
  • File Validation

Extras

  • Multiple Files
  • Securing Uploads
  • Testing with Shrine

Migrating

  • Managing Derivatives
  • Migrating File Locations
  • Migrating File Storage

Extending

  • Writing a Plugin
  • Writing a Persistence Plugin
  • Writing a Storage

Upgrading

  • Upgrading to Shrine 3.x
  • Upgrading from CarrierWave
  • Upgrading from Paperclip
  • Upgrading from Refile
Edit

Migrating File Storage

This guides shows how to move file attachments to a different storage in production, with zero downtime.

Let's assume we have a Photo model with an image file attachment stored in AWS S3 storage:

Shrine.storages = { 
  cache: Shrine::Storage::S3.new(...),
  store: Shrine::Storage::S3.new(...),
}
 
Shrine.plugin :activerecord
class ImageUploader < Shrine
  # ... 
end
class Photo < ActiveRecord::Base
  include ImageUploader::Attachment(:image)
end

Let's also assume that we're migrating from AWS S3 to Google Cloud Storage, and we've added the new storage to Shrine.storages:

Shrine.storages = { 
  ...
  store: Shrine::Storage::S3.new(...),
  gcs:   Shrine::Storage::GoogleCloudStorage.new(...),
}

1. Mirror upload and delete operations

The first step is to start mirroring uploads and deletes made on your current storage to the new storage. We can do this by loading the mirroring plugin:

Shrine.plugin :mirroring, mirror: { store: :gcs }

Put the above code in an initializer and deploy it.

You can additionally delay the mirroring into a background job for better performance.

2. Copy the files

Next step is to copy all remaining files from current storage into the new storage using the following script. It fetches the photos in batches, downloads the image, and re-uploads it to the new storage.

Photo.find_each do |photo|
  attacher = photo.image_attacher
 
  next unless attacher.stored?
 
  attacher.file.trigger_mirror_upload
 
  # if using derivatives 
  attacher.map_derivative(attacher.derivatives) do |_, derivative|
    derivative.trigger_mirror_upload
  end
end

Now the new storage should have all files the current storage has, and new uploads will continue being mirrored to the new storage.

3. Update storage

Once all the files are copied over to the new storage, everything should be ready for us to update the storage in the Shrine configuration. We can keep mirroring, in case the change would need to reverted.

Shrine.storages = { 
  ...
  store: Shrine::Storage::GoogleCloudStorage.new(...),
  s3:    Shrine::Storage::S3.new(...),
}
 
Shrine.plugin :mirroring, mirror: { store: :s3 } # mirror to :s3 storage 

4. Remove mirroring

Once everything is looking good, we can remove the mirroring:

Shrine.storages = {
  ...
  store: Shrine::Storage::GoogleCloudStorage.new(...),
- s3:    Shrine::Storage::S3.new(...),
}
 
- Shrine.plugin :mirroring, mirror: { store: :s3 } # mirror to :s3 storage
← Migrating File LocationsWriting a Plugin →
  • 1. Mirror upload and delete operations
  • 2. Copy the files
  • 3. Update storage
  • 4. Remove mirroring
Shrine
Docs
GuidesPluginsExternalContributing
Community
DiscourseStack Overflow
More
BlogGitHubStar
Follow @shrine_rb
Copyright © 2022 Janko Marohnić