-
[x] Prep
- [x] Acquire new server
- [x] Install & configure OS (Alpine Linux)
- [x] Configure external networking
- [x] Configure Docker Swarm
- [x] Configure management backplane
- [x] Transfer Docker Swarm data
- [x] Configs
- [x] Secrets
- [x] Stacks
- [x] Sync CDN content
- [x] Add DNS records for new environment
-
[x] Action
- [x] Bring down old environment
- [x] Update DNS records to point to new environment
- [x] Configure HAProxy on old environment to forward traffic to new environment
- [x] Migrate volumes
- [x] DB - Live
- [x] DB - Replica
- [x] Redis
- [x] Bring up services
- [x] DB
- [x] Redis
- [x] Streaming API
- [x] Redis (ephemeral cache instance)
- [x] Sync CDN - Final
- [x] Bring up services (cont.)
- [x] CDN
- [x] Web
- [x] Sidekiq
- [x] Default
- [x] Mail
- [x] Pull
- [x] Push
- [x] Scheduled
- [x] Maintenance
- [x] Migrate volumes (cont.)
- [x] DB backup configuration
- [x] DB backup generation working-area
- [x] Bring up services (cont.)
- [x] DB backups (schema export, DB export, archiver)
- [x] DB offsite sync
- [x] DB stale data pruner
- [x] Final smoke-test
- [x] Announce completion
-
[x] Post
- [x] Monitor traffic on old environment until none remains
- [x] Shut down docker hosts
- [x] Take off-site backup of VM disks
- [x] Secure-erase VM disk files
- [x] Move other infrastructure off
- [x] ns2.maff.scot
- [x] bark.pup.cloud
- [x] commit.pup.cloud
- [x] InfluxDB
- [x] Invidious
- [x] IRC client
- [x] Node-RED
- [x] Instance A
- [x] Instance B
- [x] Shut down those docker hosts
- [x] Shut down NFS server
- [x] Shut down firewall VM
- [x] Take off-site backup of firewall and NFS volumes
- [x] Secure-erase those VM disks
- [x] Take off-site backup of hypervisor data
- [x] Remove temporary firewall rule for old environment’s management IP which granted access to a migration service on the new environment
- [x] Reboot into a netboot recovery environment
- [x] Secure-erase both HDDs
- [x] Cancel additional IPs
- [x] Cancel server
- [x] Remove stale DNS entries
Hitches
- [x] Fix missing mail config in sk_default - caused transactional mail outage post-3.4.4 upgrade
- [x] Add SMTP_ environment variables to sidekiq default queue runner temporarily
- [x] Restore commit.pup.cloud
- [x] Push fix to
libloader
to add enrich mail
to role sidekiq
sub-role main
- [x] Rebuild docker image
maffsie/qp-mastodon:v3.4.4
and :latest
- [x] Remove direct ENV declarations for docker service
mastodon_sk_default