Enabling Wasabi Object Storage for Mastodon Media

This post is a continuation of this post

It depends on the first one.

1. Wasabi Config

Since I'm running my Baremetal in OVH Canada, I'll create my bucket in the same country.

Wasabi CA Central 1 (Toronto)	s3.ca-central-1.wasabisys.com

1.1 create a bucket mastodon-prod-media

Add this polity to the bucket policy configuration

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AddPerm",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::mastodon-prod-media/*"
    }
  ]
}

1.2 create a user mastodon-prod-user

  1. Create an user with Programmatic API Key.
  2. Save the key in your password manager.

1.3 create a policy mastodon-prod-policy

Create a policy with the content bellow

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::mastodon-prod-media"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::mastodon-prod-media/*"
    }
  ]
}

1.4 edit the user and add the police

Go to the url

Configure the policy

  1. select the policies tab
  2. select the mastodon-prod-policy

Done!

2. Mastodon Config

2.1 Sync static files

install the aws-cli

apt install python3-pip
pip install awscli

configure your aws-credentials

$aws configure
AWS Access Key ID [None]: XXXXXXXXXXXX
AWS Secret Access Key [None]: XXXXXXXXXXXXXXXXXX
Default region name [None]:
Default output format [None]:

copy the existing files to the object storage

cd /opt/mastodon/data
screen
aws s3 sync system/ s3://mastodon-prod-media/ --endpoint-url=https://s3.ca-central-1.wasabisys.com

2.2 Mastodon Configuration

Edit the application.env file

vim /etc/mastodon/docker/application.env

You will change this on the application.env

# rails will serve static files?

RAILS_SERVE_STATIC_FILES=false

And you will add this section to the application.env

# File storage (optional)
# -----------------------
S3_ENABLED=true
S3_BUCKET=mastodon-prod-media
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_HERE
AWS_SECRET_ACCESS_KEY=YOUR_ACCESS_SECRET_KEY_HERE
S3_HOSTNAME=media.gcn.sh
S3_PROTOCOL=https
S3_ENDPOINT=https://s3.ca-central-1.wasabisys.com
S3_FORCE_SINGLE_REQUEST=true

Restart your mastodon stack.

docker-compose down
docker-compose up -d

Done!

3. NGINX Config

We'll use a NGINX in front of our mastodon to have a friendly URL for our media files and a cache layer to improve the performance.

proxy_cache_path /var/cache/mastodon-prod-media levels=1:2 keys_zone=mastodon_media:100m max_size=2g inactive=12h;

server {
    listen your_listen_ip_here:80;
    server_name media.gcn.sh;
    return 301 https://media.gcn.sh$request_uri;

    access_log /dev/null;
    error_log /dev/null;
}

server {
    listen your_listen_ip_here:443 ssl http2;
    server_name media.gcn.sh;

    access_log /var/log/nginx/media-mastodon-access.log;
    error_log /var/log/nginx/media-mastodon-error.log;

  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
  ssl_prefer_server_ciphers on;
  ssl_session_cache shared:SSL:10m;
  ssl_session_tickets off;

  ssl_certificate /etc/letsencrypt/live/gcn.sh/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/gcn.sh/privkey.pem;

    location /mastodon-prod-media/ {
        proxy_cache mastodon_media;
        proxy_cache_revalidate on;
        proxy_buffering on;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;
        proxy_cache_lock on;
        proxy_cache_valid 1d;
        proxy_cache_valid 404 1h;
        proxy_ignore_headers Cache-Control;
        add_header X-Cached $upstream_cache_status;
        proxy_pass https://s3.us-central-1.wasabisys.com/mastodon-prod-media/;
    }
}

It'll cache the requests for about 12h and we're limiting the disk usage to 2 GB.

3.1 NGINX cache validation

Go to your account, copy a image URL address and use CURL againt the URL.

curl -I https://media.gcn.sh/mastodon-prod-media/media_attachments/cache/media_attachments/files/110/220/330/440/550/660/small/filename.jpeg

On the first try need to get a MISS on the x-cached header response

x-cached: MISS

On the second try, you need to get a HIT on the x-cached header response

x-cached: HIT

If you get the MISS and the HIT after, everything is fine.

If you get the HIT, it's ok too :)

4. Cleanup session

  1. Run the sync one more time, just to be sure
  2. Remove the contents of the directory /opt/mastodon/data/web/system
rm -rf /opt/mastodon/data/web/system/*

4.1 Why am I removing this?

Your media files are now on the Wasabi Object Storage, you don't need the local files anymore.

5. References

Did you like our content?

We have a lot to share; visit our site!

Our fediverse services ;)

Chat and video? We have it!

Translation tools

Video Platform Frontends

Text Editors

You can also visit our hacking space!

Follow our founder!

Follow the status of our tools

Do you want to support us? You can!

See you!

[s]