saint

We recently upgraded our Mastodon instance from v4.4.4 to v4.5.0, bumping the Helm chart from 6.5.3 to 6.6.0. While the upgrade itself was straightforward, we encountered an interesting challenge that's worth sharing.

What's New in Mastodon v4.5.0?

The v4.5.0 release brings some exciting features:

• ✨ Quote Posts – Full support for authoring and displaying quotes
• 🔄 Dynamic Reply Fetching – Better conversation threading in the web UI
• 🚫 Username Blocking – Server-wide username filtering
• 🎨 Custom Emoji Overhaul – Complete rendering system rewrite
• 📊 Enhanced Moderation Tools – Improved admin/moderator interface
• ⚡ Performance Improvements – Optimized database queries

The Architecture Gotcha

Everything seemed perfect during the upgrade process. We:

1. Merged the upstream chart cleanly
2. Updated our custom configurations (LibreTranslate, Sentry integration)
3. Built new Docker images with Sentry monitoring
4. Pushed to our registry

But when we deployed, pods started crashing with a cryptic error:

exec /usr/local/bundle/bin/bundle: exec format error

The Root Cause

The issue? Architecture mismatch. We built our Docker images on an ARM64 Mac (Apple Silicon), but our Kubernetes cluster runs on AMD64 (x86_64) nodes. The images were perfectly valid—just for the wrong architecture!

The Fix

The solution was simple but important:

docker build --platform linux/amd64 \
  -f Dockerfile.mastodon-sentry \
  -t registry.example.com/mastodon-sentry:v4.5.0 \
  . --push

By explicitly specifying --platform linux/amd64, Docker builds images compatible with our cluster architecture, even when building on ARM64 hardware.

We updated our build script to always include this flag, preventing future issues:

# Build for AMD64 (cluster architecture)
docker build --platform linux/amd64 \
  -f Dockerfile.mastodon-sentry \
  -t ${REGISTRY}/mastodon-sentry:${VERSION} \
  . --push

Deployment Results

After rebuilding with the correct architecture:

• ✅ All pods running healthy (web, streaming, sidekiq)
• ✅ Elasticsearch cluster rolled out successfully
• ✅ PostgreSQL remained stable
• ✅ Zero data loss, minimal downtime
• ✅ All customizations preserved (LibreTranslate, Sentry, custom log levels)

Deployment Stats: – Total time: ~2 hours (including troubleshooting) – Downtime: Minimal (rolling update)

Lessons Learned

1. Always specify target platform when building images for deployment, especially in cross-architecture development environments
2. Build scripts should be architecture-aware to prevent silent failures
3. Test deployments catch issues early – the error appeared immediately during pod startup
4. Keep customizations isolated – Our values-river.yaml approach made the upgrade smooth

What's Next?

We still need to reindex Elasticsearch for the new search features:

kubectl exec -n mastodon deployment/river-mastodon-web -- \
  tootctl search deploy

This will update search indices for all accounts, statuses, and tags to take advantage of v4.5.0's improved search capabilities.

Key Takeaway

Modern development often happens on ARM64 Macs while production runs on AMD64 servers. Docker's multi-platform build support makes this seamless—but only if you remember to use it! A simple --platform flag saved us from a much longer debugging session.

Happy federating! 🐘

---

Resources: – Mastodon v4.5.0 Release Notes – Docker Multi-Platform Builds – Mastodon Helm Chart

---

Our instance runs on Kubernetes with custom Sentry monitoring, LibreTranslate integration, and an alternative Redis implementation. All chart templates remain identical to upstream, with customizations isolated in our values file.

Comment in the Fediverse @saint@river.group.lt