Installing mastodon 4.1.2 using docker on Ubuntu

Updates

Last Update:

2023-07-07

2023-06-19

Introduction

This post is a Howto to install Mastodon 4.1.2 using Docker.

We hope you can install Mastodon 4.1.2 using docker like we did.

This is almost the same setup we use to run the instance https://bolha.us.

Information Section

Base Linux System

Ubuntu 20.04 or higher, always.

We're running a Virtual Machine (KVM) inside an open-source Hypervisor.

Hardware size

This proposal uses only docker to handle 500 active users in a 1500 registered server running on a single node (KVM).

Partition layout

OPT reserves

Our Baremetal Provider

We have our own BareMetal Server with ProxMox 7.

We have several Virtual Machines running different Fediverse Tools.

Our VPS Providers

We use OVH/Canada VPS for

We use VULTR/VPS

Other Providers

OnPrem Services

PreReqs Section

IPTABLES Config

if you are running nginx on the same machine

if you are running nginx externally

in your nginx server (if it's dedicated to mastodon)

Fail2ban Config

Use it

App-armor

First, It's essential to have it up and running continuously.

However, it can cause abnormal behaviors in some scenarios. It's best to keep this component disabled – during the installation, especially if you don't know how to use it or how to configure profiles in case of a problem between docker, mastodon, and app-armor.

It usually won't interfere with the docker or mastodon configuration, but, in case of problems with aa-profiles, we won't cover the solution here. It's best for you to disable it during the installation, and you can re-enable it after, if you want, and know what you are doing.

During the how-to validation, the default Ubuntu app-armor config was enabled, and everything worked fine. However, it's important to mention this in the case of different app-armor configs.

Installation section

1. Installing docker

installing

curl https://get.docker.com | bash

enabling

systemctl enable docker

starting

systemctl start docker

2. Installing docker-compose

installing

curl -s https://api.github.com/repos/docker/compose/releases/latest | grep browser_download_url  | grep docker-compose-linux-x86_64 | cut -d '"' -f 4 | wget -qi -

enabling

chmod +x docker-compose-linux-x86_64

moving to /usr/local/bin, make sure that the dir it's in the PATH var.

mv docker-compose-linux-x86_64 /usr/local/bin/docker-compose

3. Creating directories

main dirs

mkdir -p /opt/mastodon/
mkdir -p /opt/mastodon/{docker,data}

sub-dirs

mkdir -p /opt/mastodon/data/{app,web,database}
mkdir -p /opt/mastodon/data/database/{postgresql,redis,elasticsearch}
mkdir -p /opt/mastodon/data/web/{public,system,config,app}

4. Configuring permissions

creating user and groups

groupadd -g 991 mastodon
useradd mastodon -u 991 -g 991

fixing web perms

chown -R mastodon:mastodon /opt/mastodon/data/web
chown -R mastodon:mastodon /opt/mastodon/data/web/config
chown -R mastodon:mastodon /opt/mastodon/data/web/public
chown -R mastodon:mastodon /opt/mastodon/data/web/system
chown -R mastodon:mastodon /opt/mastodon/data/web/app

fixing database perms

chown -R 1000:1000 /opt/mastodon/data/database/elasticsearch

5. Creating docker config

create the file

vim /opt/mastodon/docker/docker-compose.yml

content

version: '3'

services:
  postgresql:
    image: "postgres:${POSTGRESQL_VERSION}"
    container_name: mastodon_postgresql
    restart: always
    env_file: 
      - database.env
      - versions.env
    shm_size: 256mb
    healthcheck:
      test: ['CMD', 'pg_isready', '-U', 'postgres']
    volumes:
      - postgresql:/var/lib/postgresql/data
    networks:
      - internal_network

  redis:
    image: "redis:${REDIS_VERSION}"
    container_name: mastodon_redis
    restart: always
    env_file: 
      - database.env
      - versions.env
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    volumes:
      - redis:/data
    networks:
      - internal_network

  redis-volatile:
    image: "redis:${REDIS_VERSION}"
    container_name: mastodon_redis_cache
    restart: always
    env_file: 
      - database.env
      - versions.env
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
    networks:
      - internal_network

  elasticsearch:
    image: "elasticsearch:${ELASTICSEARCH_VERSION}"
    container_name: mastodon_elastisearch
    restart: always
    env_file: 
      - database.env
      - versions.env
    environment:
      - cluster.name=elasticsearch-mastodon
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - ingest.geoip.downloader.enabled=false
    ulimits:
      memlock:
        soft: -1
        hard: -1
    healthcheck:
      test: ["CMD-SHELL", "nc -z elasticsearch 9200"]
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
    networks:
      - internal_network

  website:
    image: "tootsuite/mastodon:${MASTODON_VERSION}"
    container_name: mastodon_website
    env_file: 
      - application.env
      - database.env
      - versions.env
    command: bash -c "bundle exec rails s -p 3000"
    restart: always    
    depends_on:
      - postgresql
      - redis
      - redis-volatile
      - elasticsearch
    ports:
      - '3000:3000'
    networks:
      - internal_network
      - external_network
    healthcheck:
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:3000/health || exit 1']
    volumes:
      - public:/opt/mastodon/public
      - uploads:/opt/mastodon/public/system
      - app:/opt/mastodon/app
      - config:/opt/mastodon/config
       
  streaming:
    image: "tootsuite/mastodon:${MASTODON_VERSION}"
    container_name: mastodon_streaming
    env_file: 
      - application.env
      - database.env
      - versions.env
    command: node ./streaming
    environment:
      - DB_POOL=4
    restart: always
    depends_on:
      - postgresql
      - redis
      - redis-volatile
      - elasticsearch
    ports:
      - '4000:4000'
    networks:
      - internal_network
      - external_network
    healthcheck:
      test: ['CMD-SHELL', 'wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health || exit 1']
    volumes:
      - public:/opt/mastodon/public
      - uploads:/opt/mastodon/public/system
      - app:/opt/mastodon/app
      - config:/opt/mastodon/config


  sidekiq:
    image: "tootsuite/mastodon:${MASTODON_VERSION}"
    container_name: mastodon_sidekiq
    env_file: 
      - application.env
      - database.env
      - versions.env
    restart: always
    depends_on:
      - postgresql
      - redis
      - redis-volatile
      - website
    networks:
      - internal_network
      - external_network
    environment:
      - DB_POOL=18
    healthcheck:
      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
    command: bundle exec sidekiq -c 18
    volumes:
      - public:/opt/mastodon/public
      - uploads:/opt/mastodon/public/system
      - app:/opt/mastodon/app
      - config:/opt/mastodon/config

  shell:
    image: "tootsuite/mastodon:${MASTODON_VERSION}"
    env_file: 
      - application.env
      - database.env
      - versions.env
    command: /bin/bash 
    restart: "no"
    networks:
      - internal_network
      - external_network
    volumes:
      - public:/opt/mastodon/public
      - uploads:/opt/mastodon/public/system
      - app:/opt/mastodon/app
      - config:/opt/mastodon/config

networks:
  external_network:
  internal_network:
    internal: true

volumes:
  postgresql:
    driver_opts:
      type: none
      device: /opt/mastodon/data/database/postgresql
      o: bind    
  redis:
    driver_opts:
      type: none
      device: /opt/mastodon/data/database/redis
      o: bind    
  elasticsearch:
    driver_opts:
      type: none
      device: /opt/mastodon/data/database/elasticsearch
      o: bind    
  uploads:
    driver_opts:
      type: none
      device: /opt/mastodon/data/web/system
      o: bind
  app:
    driver_opts:
      type: none
      device: /opt/mastodon/data/web/app
      o: bind
  config:
    driver_opts:
      type: none
      device: /opt/mastodon/data/web/config
      o: bind
  public:
    driver_opts:
      type: none
      device: /opt/mastodon/data/web/public
      o: bind

6. Env Files

6.1 versions.env

create the versions.env file

vim /opt/mastodon/docker/versions.env

content

MASTODON_VERSION=v4.1.2
POSTGRESQL_VERSION=14
ELASTICSEARCH_VERSION=7.17.10
REDIS_VERSION=7

creating a symbolic link to load the versions properly

cd /opt/mastodon/docker; ln -s versions.env .env

6.2 application.env

create the application.env file

vim /opt/mastodon/docker/application.env

content

# environment config

RAILS_ENV=production
NODE_ENV=production

# web performance/tuning/concurrency

WEB_CONCURRENCY=2
MAX_THREADS=4

# locale config

DEFAULT_LOCALE=en

# local domain of your instance

LOCAL_DOMAIN=dev.bolha.us

# redirect to the first profile?

SINGLE_USER_MODE=false

# rails will serve static files?

RAILS_SERVE_STATIC_FILES=true

# email config

SMTP_SERVER=smtp.provider.tld
SMTP_PORT=587
SMTP_LOGIN=mastodon@provider.tld
SMTP_AUTH_METHOD=plain
SMTP_FROM_ADDRESS=mastodon@provider.tld
SMTP_PASSWORD=change_this_password_to_the_real_one

# secrets

SECRET_KEY_BASE=YOU_WILL_GENERATE_AND_REPLACE_HERE_LATER_CONTINUE_THE_DOC
OTP_SECRET=YOU_WILL_GENERATE_AND_REPLACE_HERE_LATER_CONTINUE_THE_DOC

# web push

VAPID_PRIVATE_KEY=YOU_WILL_GENERATE_AND_REPLACE_HERE_LATER_CONTINUE_THE_DOC
VAPID_PUBLIC_KEY=YOU_WILL_GENERATE_AND_REPLACE_HERE_LATER_CONTINUE_THE_DOC

6.3 database.env

create the database.env file

vim /opt/mastodon/docker/database.env

content

# postgresql config

POSTGRES_HOST=postgresql
POSTGRES_USER=mastodon
POSTGRES_DB=mastodon_production
POSTGRES_PASSWORD=you_will_change_this_password_ahead_on_this_doc

# elasticsearch config

ES_JAVA_OPTS="-Xms1024m -Xmx2048m"
ELASTIC_PASSWORD=you_will_change_this_password_ahead_on_this_doc

# redis config

REDIS_HOST=redis
REDIS_PORT=6379
REDIS_URL=redis://redis:6379

# redis cache config

CACHE_REDIS_HOST=redis-volatile
CACHE_REDIS_PORT=6379
CACHE_REDIS_URL=redis://redis-volatile:6379

# postgresql mastodon integration

DB_HOST=postgresql
DB_USER=mastodon
DB_NAME=mastodon_production
DB_PASS=you_will_change_this_password_ahead_on_this_doc
DB_PORT=5432

# elasticsearch mastodon integration

ES_ENABLED=true
ES_HOST=elasticsearch
ES_PORT=9200
ES_USER=elastic
ES_PASS=you_will_change_this_password_ahead_on_this_doc

7. generating secrets and passwords

7.1 secret key and OTP secret

Now we need to generate two secrets, SECRETKEYBASE and OTP_SECRET.

From the docker host run

$ docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell bundle exec rake secret

Yes, you need to run two times, one for each secret and then update the application.env file.

7.2 generate vapid secrets

Now we need to generate the VAPID secrets, VAPIDPRIVATEKEY and VAPIDPUBLICKEY.

From the docker host run

$ docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell bundle exec rake mastodon:webpush:generate_vapid_key

Get the output and append the file application.env

7.3 generate elasticsearch password

Generate a password for the var ELASTIC_PASSWORD

$ openssl rand -hex 15

Update the database.env file, don't forget to update ES_PASS, it's the same password.

7.4 generate postgresql password

Generate a password for the var POSTGRES_PASSWORD

$ openssl rand -hex 15

Update the database.env file.

8. configuring a local nginx to serve mastodon

Here we'll configure an NGINX on the same docker host of our mastodon.

8.1 extracting the static files for nginx cache

This procedure is only used by NGINX for CACHE purposes, mastodon do not use this static files, it's used by NGINX, and NGINX Only.

let's create a docker volume in YOUR NGINX SERVER to store the static mastodon files

$ mkdir -p /opt/www/mastodon/dev.bolha.us/public/4.1.2
$ chown -R mastodon:mastodon /opt/www/mastodon
$ docker volume create --opt type=none --opt device=/opt/www/mastodon/dev.bolha.us/public/4.1.2 --opt o=bind mastodon_public_4.1.2

now let's copy the files from the container to the device

$ docker run --rm -v "mastodon_public_4.1.2:/static" tootsuite/mastodon:v4.1.2 bash -c "cp -r /opt/mastodon/public/* /static/"

nice, check if the files were copied properly

$ ls /opt/www/mastodon/dev.bolha.us/public/4.1.2

output expected

500.html  avatars    embed.js  favicon.ico  inert.css  oops.gif  packs       sounds  sw.js.map                 web-push-icon_favourite.png
assets    badge.png  emoji     headers      ocr        oops.png  robots.txt  sw.js   web-push-icon_expand.png  web-push-icon_reblog.png

now we can remove our docker volume; we only needed the volume for the copy.

$ docker volume rm mastodon_public_4.1.2

ok, now we have a directory to be used by nginx cache to serve the static files.

8.2 Creating directories

$ mkdir -p /opt/nginx/{docker,html,vhost,conf,stream,certbot,cache,logs}
$ mkdir -p /opt/nginx/certbot/{html,conf}
$ mkdir -p /opt/nginx/cache/public/4.1.2

8.3 Creating nginx.conf

Let's create our nginx.conf

$ vim /opt/nginx/conf/nginx.conf

The contents of the file

user nginx;
worker_processes auto;

pid        /var/run/nginx.pid;

include /etc/nginx/conf.d/*.conf;

events {
	worker_connections 2048;
}

http {

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;

	keepalive_timeout 65;
	types_hash_max_size 2048;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;
	
	log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	gzip on;

	include /etc/nginx/vhosts/*.conf;
	include /etc/nginx/sites-enabled/*;

}

stream {

 log_format proxy '$remote_addr [$time_local] '
                 '$protocol $status $bytes_sent $bytes_received '
                 '$session_time "$upstream_addr" '
                 '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time" "$upstream_addr"';

   include /etc/nginx/stream/*.conf;

}

8.4 Creating default.conf

Let's create our default.conf to avoid nginx container default configs conflict.

$ vim /opt/nginx/vhost/default.conf

The contents of the file

server {
    listen       80;
    server_name  localhost;

    location /.well-known/acme-challenge/ {
      root /var/www/certbot;
    }
  
    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}

8.5 Creating dev.bolha.us.conf

Now we can create the config of our mastodon instance; we will use dev.bolha.us as our domain, just as an example.

$ vim /opt/nginx/vhost/dev-bolha-us.conf

With this content

# connection configuration

map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;
}

# upstream configuration

upstream backend {
    server localhost:3000 fail_timeout=0;
}

upstream streaming {
    server localhost:4000 fail_timeout=0;
}

# cache for static files

proxy_cache_path /var/cache/mastodon/public/4.1.2 levels=1:2 keys_zone=MASTODON_CACHE_v412:10m inactive=7d max_size=3g;

# server configuration

server {
  listen 80;
  server_name dev.bolha.us;

  location /.well-known/acme-challenge/ {
      root /var/www/certbot;
  }
  
  location / { 
    return 301 https://dev.bolha.us$request_uri; 
  }
  
}

server {
  listen 443 ssl http2;
  server_name dev.bolha.us;

  ssl_protocols TLSv1.2 TLSv1.3;
  ssl_prefer_server_ciphers on;
  ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';

  ssl_session_cache shared:SSL:10m;
  ssl_session_tickets off;

  access_log /var/log/nginx/mastodon-dev-bolha-us-access.log;
  error_log /var/log/nginx/mastodon-dev-bolha-us-error.log;

  ssl_certificate /etc/letsencrypt/live/bolha.us/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/bolha.us/privkey.pem;

  keepalive_timeout    70;
  sendfile             on;
  client_max_body_size 80m;

  root /var/www/mastodon/dev.bolha.us/public/4.1.2;

  gzip on;
  gzip_disable "msie6";
  gzip_vary on;
  gzip_proxied any;
  gzip_comp_level 6;
  gzip_buffers 16 8k;
  gzip_http_version 1.1;
  gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml image/x-icon;

  add_header Strict-Transport-Security "max-age=31536000" always;

  location / {
    try_files $uri @proxy;
  }

  location ~ ^/(system/accounts/avatars|system/media_attachments/files) {
    add_header Cache-Control "public, max-age=31536000, immutable";
    add_header Strict-Transport-Security "max-age=31536000" always;
    root /opt/mastodon/;
    try_files $uri @proxy;
  }

  location ~ ^/(emoji|packs) {
    add_header Cache-Control "public, max-age=31536000, immutable";
    add_header Strict-Transport-Security "max-age=31536000" always;
    try_files $uri @proxy;
  }

  location /sw.js {
    add_header Cache-Control "public, max-age=0";
    add_header Strict-Transport-Security "max-age=31536000" always;
    try_files $uri @proxy;
  }

  location @proxy {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Proxy"";
    proxy_pass_header Server;

    proxy_pass http://backend;
    proxy_buffering on;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    proxy_cache MASTODON_CACHE_v412;
    proxy_cache_valid 200 7d;
    proxy_cache_valid 410 24h;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    add_header X-Cached $upstream_cache_status;
    add_header Strict-Transport-Security "max-age=31536000" always;

    tcp_nodelay on;
  }

  location /api/v1/streaming {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Proxy"";

    proxy_pass http://streaming;
    proxy_buffering off;
    proxy_redirect off;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    tcp_nodelay on;
  }

  error_page 500 501 502 503 504 /500.html;
}

What do you need to change here?

To use this config, we expect that you have it before starting your nginx

Wait and follow the instructions carefully, baby steps!

8.6 Creating the nginx docker-compose configuration

After that, we can create the docker-compose file.

cd /opt/nginx/docker
vim docker-compose.yml

here are the contents of the file

version: '3'

services:
  nginx:
    image: nginx:latest
    container_name: nginx
    restart: always
    network_mode: host
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /opt/nginx/conf/nginx.conf:/etc/nginx/nginx.conf
      - nginx_log:/var/log/nginx
      - nginx_vhost:/etc/nginx/vhosts
      - nginx_stream:/etc/nginx/stream
      - nginx_html:/usr/share/nginx/html
      - nginx_cache:/var/cache/mastodon
      - certbot_conf:/etc/letsencrypt
      - certbot_html:/var/www/certbot
      - mastodon_public:/var/www/mastodon
    healthcheck:
      test: ["CMD-SHELL", "wget -O /dev/null http://localhost || exit 1"]
      timeout: 10s
  certbot:
    image: certbot/certbot:latest
    restart: no
    container_name: certbot
    network_mode: host
    volumes:
      - certbot_conf:/etc/letsencrypt
      - certbot_html:/var/www/certbot

volumes:
  mastodon_public:
    driver_opts:
      type: none
      device: /opt/www/mastodon
      o: bind
  nginx_log:
    driver_opts:
      type: none
      device: /opt/nginx/logs
      o: bind
  nginx_cache:
    driver_opts:
      type: none
      device: /opt/nginx/cache
      o: bind
  nginx_vhost:
    driver_opts:
      type: none
      device: /opt/nginx/vhost
      o: bind
  nginx_stream:
    driver_opts:
      type: none
      device: /opt/nginx/stream
      o: bind
  nginx_html:
    driver_opts:
      type: none
      device: /opt/nginx/html
      o: bind
  certbot_conf:
    driver_opts:
      type: none
      device: /opt/nginx/certbot/conf
      o: bind
  certbot_html:
    driver_opts:
      type: none
      device: /opt/nginx/certbot/html
      o: bind

8.7 Creating the letsencrypt certificate

let's create the certificate first. It would be best to ensure that your domain points to your nginx docker server or cerbot will fail during the certificate generation.

$ cd /opt/nginx/docker
$ docker-compose run --rm certbot certonly --webroot --webroot-path /var/www/certbot/ -d dev.bolha.us

add the command to renew to your crontab

$ crontab -e

add this command

00 3 * * * docker-compose /opt/nginx/docker/docker-compose.yml run --rm certbot renew

8.8 Starting NGINX

now let's run our nginx :)

$ cd /opt/nginx/docker
$ docker-compose -up -d

check if your nginx is running accordingly

$ docker ps
$ docker logs -f

9. Configuring an external NGINX dedicated to mastodon

Point the domain to the external nginx server.

Follow section 8 in your NGINX Server.|

The static files are only needed by NGINX, you do not need to do that part on the mastodon server.

Open the ports 3000 and 4000 TCP for the NGINX IP on your Mastodon Server.

10. Configuring an existing (external) non-docker NGINX

Follow the instructions from the sections:

You need to create the cache directory for nginx

proxy_cache_path /var/cache/mastodon/public/4.1.2 levels=1:2 keys_zone=MASTODON_CACHE_v412:10m inactive=7d max_size=3g;

You need to generate your certs.

You need to adjust to having it running in your setup, we don't know your structure, so we can't cover this here, sorry.

11. Starting your Mastodon (Finally, right? :)

If you get here, you're already a warrior :)

Let's do it!

Go to your mastodon server docker config directory.

$ cd /opt/mastodon/docker

11.1 pulling the images

pulling

$ docker-compose pull

output expected

 ✔ postgresql Pulled                                                                                
 ✔ redis  Pulled
 ✔ redis-volatile Skipped - Image is already being pulled by redis                                                                                                     
 ✔ elasticsearch Pulled
 ✔ website Pulled                                                                                                                                                    
 ✔ streaming Skipped - Image is already being pulled by website                                                                                                       
 ✔ sidekiq Skipped - image is already being pulled by website
✔ shell Skipped - Image is already being pulled by website

11.2 starting postgresql and redis

starting databases

$ docker-compose up -d postgresql redis redis-volatile

output expected

 ✔ Network docker_internal_network  Created                                                                                                                            0.0s
 ✔ Network docker_external_network  Created                                                                                                                            0.0s
 ✔ Volume "docker_elasticsearch"    Created                                                                                                                            0.0s
 ✔ Volume "docker_public"           Created                                                                                                                            0.0s
 ✔ Volume "docker_uploads"          Created                                                                                                                            0.0s
 ✔ Volume "docker_app"              Created                                                                                                                            0.0s
 ✔ Volume "docker_postgresql"       Created                                                                                                                            0.0s
 ✔ Volume "docker_redis"            Created                                                                                                                            0.0s
 ✔ Container mastodon_redis         Started                                                                                                                            0.5s
 ✔ Container mastodon_redis_cache   Started                                                                                                                            0.5s
 ✔ Container mastodon_postgresql    Started                                                                                                                            0.5s

11.2 running the database setup

$ docker-compose run --rm shell bundle exec rake db:setup

output expected

Database 'mastodon_production' already exists

11.3 starting remaining services

$ docker-compose up -d

output expected

+] Running 8/8
 ✔ Container mastodon_elastisearch  Started                                                                                                                                                                                                                                            1.1s
 ✔ Container mastodon_website       Started                                                                                                                            1.6s
 ✔ Container mastodon_streaming     Started                                                                                                                            1.6s
 ✔ Container mastodon_sidekiq       Started                                                                                                                            2.2s

11.4 Checking everything

let's verify our containers

root@dev:/opt/nginx/docker# docker ps

output expected

CONTAINER ID   IMAGE                       COMMAND                  CREATED          STATUS                            PORTS                                NAMES
ac17eef8d384   nginx:latest "/docker-entrypoint.…" 6 seconds ago    Up 5 seconds (health: starting)                                        mastodon_nginx
adabce4171e2   tootsuite/mastodon:v4.1.2 "/usr/bin/tini -- bu…" 25 minutes ago   Up 25 minutes (healthy)           3000/tcp, 4000/tcp                   mastodon_sidekiq
47d8b9720fc4   tootsuite/mastodon:v4.1.2   "/usr/bin/tini -- ba…"   25 minutes ago   Up 25 minutes (healthy)           127.0.0.1:3000->3000/tcp, 4000/tcp   mastodon_website
f55bea899b31   tootsuite/mastodon:v4.1.2   "/usr/bin/tini -- no…"   25 minutes ago   Up 25 minutes (healthy)           3000/tcp, 127.0.0.1:4000->4000/tcp   mastodon_streaming
e2ffd239c210   redis:7 "docker-entrypoint.s…" 25 minutes ago   Up 25 minutes (healthy)                                                mastodon_redis_cache
3914fc3f784b   postgres:14 "docker-entrypoint.s…" 25 minutes ago   Up 25 minutes (healthy)                                                mastodon_postgresql
d02fffd8f108   elasticsearch:7.17.10       "/bin/tini -- /usr/l…"   25 minutes ago   Up 25 minutes (healthy)                                                mastodon_elastisearch
e500590b9a20   redis:7 "docker-entrypoint.s…" 25 minutes ago   Up 25 minutes (healthy)                                                mastodon_redis

11.5 Enabling registration with approval

$ cd /opt/mastodon/docker
$ docker-compose run --rm shell bin/tootctl settings registrations approved

12. Creating users via terminal

Creating an owner

$ docker-compose run --rm shell bin/tootctl accounts create gutocarvalho --email gutocarvalho@bolha.us --confirmed --role Owner
$ docker-compose run --rm shell bin/tootctl accounts approve gutocarvalho

Creating an admin

$ docker-compose run --rm shell bin/tootctl accounts create joseaugusto --email joseaaugusto@bolha.us --confirmed --role Admin
$ docker-compose run --rm shell bin/tootctl accounts approve joseaugusto

Creating an moderator

$ docker-compose run --rm shell bin/tootctl accounts create augustocarvalho --email augustocarvalho@bolha.us --confirmed --role Moderator
$ docker-compose run --rm shell bin/tootctl accounts approve augustocarvalho

Creating a normal user

$ docker-compose run --rm shell bin/tootctl accounts create arturcarvalho --email arturcarvalho@bolha.us --confirmed
$ docker-compose run --rm shell bin/tootctl accounts approve arturcarvalho

13. Accessing your mastodon!

Go to your mastodon!

https://dev.bolha.us

It's ready and should work, you just need to login.

14. Creating maintenance tasks using crontab

open your root crontab

$ crontab -e

indexing data of elasticsearch to create the 'https://dev.bolha.us/explore' content

00 */6 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl search deploy --concurrency 4

this will recount all accounts numbers of the instance daily

30 1 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl cache recount accounts --concurrency 4

this will force all users to follow special instance accounts

00 2 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl accounts follow status

00 2 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl accounts follow news

00 2 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl accounts follow tips

00 2 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl accounts follow gutocarvalho

this will clean media from external instances in our local cache, it will clean everything older than 15 days in the cache

30 2 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl media remove --days=15 --concurrency 4

this will clean local thumbnails for preview cards in our local cache; it will clean everything older than 15 days in the cache

00 3 * * * docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl preview_cards remove --days=15 --concurrency 4

this will regenerate all user feeds every Sunday

00 1 * * 0 docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl feeds clear
    
00 2 * * 0 docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl feeds build --concurrency 4

this will remove statuses without users/references every Sunday

00 3 * * 0 docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl statuses remove --days 60

References!

15. Maintenance tasks

Remember to add this before the command; you will use tootctl inside the mastodon_shell container

$ docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell COMMAND

Example

$ docker-compose -f /opt/mastodon/docker/docker-compose.yml run --rm shell tootctl accounts modify user@domain.tld --approve

I'll remove the first part of the command to make the examples easily understood.

15.1 Accounts Tasks

Reset password

$ tootctl accounts modify user@domain.tld --reset-password

Disable user

$ tootctl accounts modify user@domain.tld --disable

Approve user

$ tootctl accounts modify user@domain.tld --approve

Disable 2FA in case someone forgets the 2FA code or device

$ tootctl accounts modify user@domain.tld --disable-2fa

If you are not seeing user data from a specific user or domain

$ tootctl accounts refresh user@domain.tld

Delete

$ tootctl accounts delete user@domain.tld

15.2 Other tasks

If you are not seeing images from a specific domain

$ tootctl media refresh domain.tld --concurrency 4

Remove all accounts from a given DOMAIN without leaving behind any records. Unlike a suspension, if the DOMAIN still exists in the wild, it means the accounts could return if they are resolved again.

$ tootctl domains purge domain.tld --concurrency 4

Remove remote accounts that no longer exist. Queries every single remote account in the database to determine if it still exists on the origin server, and if it doesn't, then remove it from the database.

Accounts with confirmed activity within the last week are excluded from the checks, in case the server is down.

$ tootctl accounts cull domain.tld --concurrency 4

16. Backup

You can use whatever you want to protect your data; we'll focus on what you need to do before the backup and what you need to backup to have your data replicated to a safe location.

16.1 Cold Backup

  1. stop all containers

  2. run a backup of your /opt/mastodon, /opt/nginx, and /opt/www

16.2 Hot Backup

  1. run a backup of your postgresql

  2. run a backup of your /opt/mastodon/data/web directory

  3. run a backup of your /opt/mastodon/docker directory

  4. run a backup of your /opt/nginx directory

16.3 Our Backup Provider

Our automation sends our backup to blackblaze object storage, it's a good and cheap provider; we do full backups of pgsql and configs every day.

17. API Information

You cant get some info from the API using curl and jq

$ curl -s https://dev.bolha.us/api/v1/instance | jq
$ curl -s https://dev.bolha.us/api/v2/instance | jq

Let's create a script to get some information from mastodon:

    $ vim mastodon_api_info.sh

Script Content

#!/bin/bash

INSTANCE_ENDPOINT_V1="https://dev.bolha.us/api/v1/instance"
INSTANCE_ENDPOINT_V2="https://dev.bolha.us/api/v2/instance"
INSTANCE_URL="https://dev.bolha.us"

COUNT_TOTAL_USERS=$(curl -s $INSTANCE_ENDPOINT_V1 | jq '.stats.user_count')
COUNT_TOTAL_STATUS=$(curl -s $INSTANCE_ENDPOINT_V1 | jq '.stats.status_count')
COUNT_TOTAL_DOMAINS=$(curl -s $INSTANCE_ENDPOINT_V1 | jq '.stats.domain_count')
COUNT_ACTIVE_USERS=$(curl -s $INSTANCE_ENDPOINT_V2 | jq '.usage.users.active_month')
COUNT_POOL_LIMIT=$(curl -s $INSTANCE_ENDPOINT_V2 | jq '.configuration.polls.max_options')
COUNT_CHAR_LIMIT=$(curl -s $INSTANCE_ENDPOINT_V2 | jq '.configuration.statuses.max_characters')
INSTANCE_VERSION=$(curl -s $INSTANCE_ENDPOINT_V2 | jq '.version')

echo "Number of total registered users: $COUNT_TOTAL_USERS"
echo "Number of total statuses: $COUNT_TOTAL_STATUS"
echo "Number of total known domains: $COUNT_TOTAL_DOMAINS"
echo "Number of active users (this month): $COUNT_ACTIVE_USERS"
echo "Number of pool options: $COUNT_POOL_LIMIT"
echo "Number of char limit: $COUNT_CHAR_LIMIT"
echo "Mastodon instance version: $INSTANCE_VERSION"

Execute

$ bash mastodon_api_info.sh

Expected output (from bolha.us)

Number of total registered users: 1464
Number of total statuses: 51191
Number of total known domains: 16237
Number of active users (this month): 618
Number of pool options: 4
Number of char limit: 500
Mastodon instance version: "4.1.2"

:)

18. Upgrade

This process will cover the upgrade for minor versions only.

4.1.x to 4.1.y for example.

Here we'll cover the scenario where mastodon and nginx run on the same server.

18.1 pre-upgrade

  1. run a backup of your postgresql

  2. run a backup of your /opt/mastodon/data/web/system directory

  3. run a backup of your /opt/mastodon/docker directory

  4. run a backup of your /opt/nginx directory

  5. stop mastodon service

  6. stop nginx services

18.2 upgrading

Verify

:)

19. Final notes

I hope you can use this doc to configure your own instance like we did.

I wish to find something like this one year ago when I started my studies in the fediverse tools and Mastodon. :)

If you need assistance, you can reach me on the matrix @gutocarvalho@bolha.chat or mastodon @gutocarvalho@gcn.sh.

:)

This post inspired our post:

Their post inspired the work, and we tried to expand it the best we could.

The first version of bolha.us followed their instructions, and this new post is just to keep the information flowing and updated.

I want to thank the “sleeplessbeastie.eu” team for the excellent work; we are running in prod because of you :)

20. Next posts

1. Enabling Object Storage using Wasabi

2. LibreTranslate installation and integration

3. Individual Sidekiqs Containers For Queues

4. Observability with Prometheus + Grafana

5. Mastodon With TOR

[s] Guto

Did you like our content?

We have a lot to share; visit our site!

Our fediverse services ;)

Chat and video? We have it!

Translation tools

Video Platform Frontends

Text Editors

You can also visit our hacking space!

Follow our founder!

Follow the status of our tools

Do you want to support us? You can!

See you!

[s]