Skip to main content

Open-source alternatives guide

Self-Host Your Email with Mailu 2026

Set up a fully self-hosted email server with Mailu on Docker in 2026. Covers DNS configuration, SPF/DKIM/DMARC setup, Mailu vs Mailcow, and honest now.

·OSSAlt Team
Share:

TL;DR

You can replace Google Workspace ($6/user/month) with Mailu — a Docker-based open source mail server — and pay only server costs (~$6-12/month). It's a legitimate path for technical teams prioritizing privacy and cost control. But self-hosting email is genuinely hard: DNS misconfiguration means your emails go to spam, IP reputation takes weeks to build, and a single mistake can get your IP blacklisted. This guide walks through the full setup and helps you decide if it's worth it.

Key Takeaways

  • Mailu is a Docker-first mail server — SMTP, IMAP, webmail, spam filtering, and admin UI in one stack
  • DNS records are the hardest part — MX, SPF, DKIM, DMARC, and PTR records must all be correct for deliverability
  • IP warm-up is mandatory — Gmail and Outlook apply strict filtering to new IPs; ramp up slowly
  • Mailu vs Mailcow: Mailu = simpler, better docs; Mailcow = more features, bigger community
  • When to self-host email: privacy-critical orgs, developers learning the stack, personal domains
  • When NOT to: transactional email (use Resend/Postmark instead), teams without a sysadmin

Should You Self-Host Email?

Be honest about the trade-offs before starting.

Self-hosting email is right for:

  • Personal or organizational domains where privacy matters (no Google scanning your emails)
  • Technical teams with a dedicated sysadmin
  • Learning how email infrastructure works
  • Cost reduction at scale (10+ mailboxes)

Don't self-host email if:

  • You need transactional email (welcome emails, password resets) — use Resend or Postmark instead
  • You don't have time to maintain it (spam filters update constantly)
  • Your IP address changes or you're on a residential connection
  • You need Google Workspace's collaborative features (Docs, Calendar, Meet)

Rule of thumb: Self-host email only if you're willing to own the deliverability problem. It works, but it requires ongoing attention.


Mailu vs Mailcow: Which Self-Hosted Email Server?

The two dominant Docker-based mail servers:

FeatureMailuMailcow
Setup complexity⭐ SimplerMore complex
GitHub Stars~6,500~9,000
WebmailRoundcube includedSOGo + Roundcube
Admin UI✅ Clean✅ Feature-rich
Spam filteringRspamdRspamd
2FA
ARM supportLimited
Kubernetes support
DocumentationGoodExcellent
CommunityMediumLarge
Best forSimplicityFull-featured deployments

Recommendation: Start with Mailu if you want a simpler setup. Switch to Mailcow if you need more features or find Mailu limiting.


Part 1: Server Prerequisites

VPS Requirements

Email requires a dedicated static IP — not a shared host:

  • Minimum: 2 vCPU, 2 GB RAM, 20 GB storage
  • Recommended: Hetzner CX22 (€4.35/month) — dedicated IP included
  • Critical: Your provider must allow port 25 (SMTP). Some cloud providers block it by default. Hetzner, OVH, and most VPS providers allow it; AWS, GCP, and Azure block port 25 by default.

Check Port 25

Before installing anything, verify your VPS can send on port 25:

# Check if port 25 is open (from your VPS):
telnet smtp.gmail.com 25
# If it times out, your provider is blocking it

Reverse DNS (PTR Record)

This is non-negotiable. Your IP address must resolve back to your mail server hostname. Set this in your VPS provider's control panel:

YOUR_SERVER_IP → mail.yourdomain.com

Hetzner: Server → Networking → Edit PTR record
DigitalOcean: Networking → Domains → Add reverse record


Part 2: DNS Configuration

Before installing Mailu, configure all DNS records (these take up to 48 hours to propagate):

Required Records

# MX record — tells other servers where to deliver email
yourdomain.com.    IN  MX  10  mail.yourdomain.com.

# A record — IP of your mail server
mail.yourdomain.com.  IN  A  YOUR_SERVER_IP

# SPF — which servers can send mail for your domain
yourdomain.com.    IN  TXT  "v=spf1 ip4:YOUR_SERVER_IP ~all"

# DMARC — policy for failed authentication (start permissive)
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.com"

DKIM: Generated by Mailu after installation — add after setup.

These allow email clients (Thunderbird, Outlook) to auto-configure:

autoconfig.yourdomain.com.  IN  CNAME  mail.yourdomain.com.
autodiscover.yourdomain.com.  IN  CNAME  mail.yourdomain.com.

Part 3: Install Mailu

Generate Configuration

Use Mailu's official setup wizard at setup.mailu.io to generate your docker-compose.yml and mailu.env.

Key settings to configure:

  • Domain: yourdomain.com
  • Hostname: mail.yourdomain.com
  • TLS flavor: letsencrypt (automatic SSL)
  • Webmail: roundcube (recommended)

Download both generated files to your server:

mkdir /mailu && cd /mailu
# Download the generated files from setup.mailu.io
# Or use the config generator CLI

Deploy

cd /mailu
docker compose up -d

Mailu starts the following services:

  • front — Nginx reverse proxy + SSL termination
  • admin — Web admin interface
  • imap — Dovecot IMAP server
  • smtp — Postfix SMTP server
  • antispam — Rspamd
  • webmail — Roundcube

Create Your First Admin Account

docker compose exec admin flask mailu admin admin yourdomain.com 'SecurePassword123'
# Creates admin@yourdomain.com with admin privileges

Access the admin UI at https://mail.yourdomain.com/admin.


Part 4: Configure DKIM

DKIM keys are generated by Mailu on startup. Retrieve your public key:

docker compose exec admin flask mailu dkim yourdomain.com

This outputs a DNS TXT record like:

dkim._domainkey.yourdomain.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0..."

Add this to your DNS. After propagation (up to 48 hours), verify:

# Verify DKIM is publishing correctly:
dig TXT dkim._domainkey.yourdomain.com

Tighten DMARC After Verification

Once SPF and DKIM are working (check via mail-tester.com):

# Phase 1 — monitor only:
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=none; rua=mailto:dmarc@yourdomain.com"

# Phase 2 — quarantine (after 2 weeks of clean reports):
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=quarantine; rua=mailto:dmarc@yourdomain.com"

# Phase 3 — reject (full enforcement):
_dmarc.yourdomain.com.  IN  TXT  "v=DMARC1; p=reject; rua=mailto:dmarc@yourdomain.com"

Part 5: Test Deliverability

Before sending any real email, verify your setup:

# 1. Test DNS records:
dig MX yourdomain.com
dig TXT yourdomain.com          # Should show SPF
dig TXT dkim._domainkey.yourdomain.com  # Should show DKIM
dig TXT _dmarc.yourdomain.com   # Should show DMARC

# 2. Check blacklists (run from your server):
curl -s "https://api.hetrixtools.com/v2/blacklist-check/YOUR_SERVER_IP/"

# 3. Send a test email to mail-tester.com:
# Visit mail-tester.com, get a unique address, send to it, check your score
# Aim for 9/10 or higher before sending real email

IP Warm-Up Schedule

Gmail and Outlook treat new IPs with heavy suspicion. Follow this ramp-up:

Week 1:  10-50 emails/day (to engaged recipients only)
Week 2:  50-200 emails/day
Week 3:  200-500 emails/day
Week 4+: Scale gradually based on bounce/spam rates

Never blast a large list from a new IP. The first emails should go to people who know you and will mark them as "not spam" if needed.


Part 6: Adding Mailboxes

Via the admin UI at https://mail.yourdomain.com/admin:

  1. Users → Add User → Set email, password, quota
  2. Aliases → Add alias (e.g., hello@admin@)
  3. Domains → Add additional domains (multi-domain setup)

Or via CLI:

# Add a user:
docker compose exec admin flask mailu user jane yourdomain.com 'Password123'

# Add an alias:
docker compose exec admin flask mailu alias info yourdomain.com jane

Connecting Email Clients

Mailu auto-generates client configuration for:

  • IMAP: mail.yourdomain.com:993 (SSL/TLS)
  • SMTP: mail.yourdomain.com:587 (STARTTLS)
  • Webmail: https://mail.yourdomain.com

For Thunderbird, the autoconfig endpoint auto-detects settings when you enter your email address.


Ongoing Maintenance

Self-hosting email requires ongoing attention:

Monthly tasks:

When updates release:

cd /mailu
docker compose pull && docker compose up -d

Backup email data:

# Backup mailboxes (Dovecot IMAP storage):
tar -czf mail-backup-$(date +%Y%m%d).tar.gz /mailu/mail/

Cost Comparison

OptionMonthly Cost (5 users)Notes
Google Workspace Starter$30/mo$6/user
Microsoft 365 Business Basic$30/mo$6/user
Fastmail$25/mo$5/user
Mailu (self-hosted)~$6-12/moServer cost only
Proton Mail Plus$50/moPrivacy-focused managed

Self-hosting saves ~$25-50/month for a 5-person team — meaningful for bootstrapped startups, but less meaningful once you factor in maintenance time.


Methodology

  • Mailu documentation: mailu.io/docs
  • Mailu GitHub: github.com/Mailu/Mailu
  • Email authentication specs: Google, DMARCLY, RFC standards
  • Comparison sources: RunCloud, GeekFlare, It's FOSS (2025-2026)

Compare all open source Google Workspace alternatives at OSSAlt.com.

Communication Tools Become Core Infrastructure Faster Than Expected

Email, chat, newsletters, and customer communication systems look lightweight until they become the delivery path for contracts, password resets, invoices, or support obligations. That is why communication tooling deserves more rigor than many self-hosting articles give it. Reliability, sender reputation, role-based access, and retention policies matter as much as UI polish. Before migrating, teams should decide whether they need transactional delivery, team collaboration, public campaigns, or knowledge sharing, because each workload has different operational constraints and failure modes.

Related services help define the right boundary. Stalwart guide matters when mail delivery and mailbox hosting are the core problem. Listmonk guide matters when the real requirement is campaign management, segments, and high-volume newsletter sending rather than general mailboxes. Outline guide often belongs nearby because communication breakdowns are frequently documentation breakdowns in disguise. A durable stack gives each tool a narrow role and avoids turning one service into a catch-all portal for every message type.

What to Validate Before You Migrate Users

The migration test should focus on user-visible failure points: SPF, DKIM, and DMARC alignment for email; permission inheritance for shared workspaces; export and archival rules for regulated environments; and mobile access for teams that do not live at desks. It is better to discover one blocked password-reset flow in staging than during a live cutover. The same goes for operations. Know how to rotate credentials, pause sending, or reroute outbound notifications before adoption.

The strongest self-hosted communication setups are boring in the best sense. They have clear DNS ownership, standard TLS handling, predictable backup routines, and a documented answer to legal hold or unsubscribe requests. Articles that emphasize those boring details help readers pick tools they can keep, not just tools they can install.

Communication Tools Become Core Infrastructure Faster Than Expected

Email, chat, newsletters, and customer communication systems look lightweight until they become the delivery path for contracts, password resets, invoices, or support obligations. That is why communication tooling deserves more rigor than many self-hosting articles give it. Reliability, sender reputation, role-based access, and retention policies matter as much as UI polish. Before migrating, teams should decide whether they need transactional delivery, team collaboration, public campaigns, or knowledge sharing, because each workload has different operational constraints and failure modes.

Related services help define the right boundary. Stalwart guide matters when mail delivery and mailbox hosting are the core problem. Listmonk guide matters when the real requirement is campaign management, segments, and high-volume newsletter sending rather than general mailboxes. Outline guide often belongs nearby because communication breakdowns are frequently documentation breakdowns in disguise. A durable stack gives each tool a narrow role and avoids turning one service into a catch-all portal for every message type.

What to Validate Before You Migrate Users

The migration test should focus on user-visible failure points: SPF, DKIM, and DMARC alignment for email; permission inheritance for shared workspaces; export and archival rules for regulated environments; and mobile access for teams that do not live at desks. It is better to discover one blocked password-reset flow in staging than during a live cutover. The same goes for operations. Know how to rotate credentials, pause sending, or reroute outbound notifications before adoption.

The strongest self-hosted communication setups are boring in the best sense. They have clear DNS ownership, standard TLS handling, predictable backup routines, and a documented answer to legal hold or unsubscribe requests. Articles that emphasize those boring details help readers pick tools they can keep, not just tools they can install.

Deliverability and Retention Notes

Communication tooling needs documented retention, account lifecycle handling, and deliverability validation. Those operational basics matter more than cosmetic interface differences once real users depend on the system.

Communication Tools Become Core Infrastructure Faster Than Expected

Email, chat, newsletters, and customer communication systems look lightweight until they become the delivery path for contracts, password resets, invoices, or support obligations. That is why communication tooling deserves more rigor than many self-hosting articles give it. Reliability, sender reputation, role-based access, and retention policies matter as much as UI polish. Before migrating, teams should decide whether they need transactional delivery, team collaboration, public campaigns, or knowledge sharing, because each workload has different operational constraints and failure modes.

Related services help define the right boundary. Stalwart guide matters when mail delivery and mailbox hosting are the core problem. Listmonk guide matters when the real requirement is campaign management, segments, and high-volume newsletter sending rather than general mailboxes. Outline guide often belongs nearby because communication breakdowns are frequently documentation breakdowns in disguise. A durable stack gives each tool a narrow role and avoids turning one service into a catch-all portal for every message type.

What to Validate Before You Migrate Users

The migration test should focus on user-visible failure points: SPF, DKIM, and DMARC alignment for email; permission inheritance for shared workspaces; export and archival rules for regulated environments; and mobile access for teams that do not live at desks. It is better to discover one blocked password-reset flow in staging than during a live cutover. The same goes for operations. Know how to rotate credentials, pause sending, or reroute outbound notifications before adoption.

The strongest self-hosted communication setups are boring in the best sense. They have clear DNS ownership, standard TLS handling, predictable backup routines, and a documented answer to legal hold or unsubscribe requests. Articles that emphasize those boring details help readers pick tools they can keep, not just tools they can install.

The SaaS-to-Self-Hosted Migration Guide (Free PDF)

Step-by-step: infrastructure setup, data migration, backups, and security for 15+ common SaaS replacements. Used by 300+ developers.

Join 300+ self-hosters. Unsubscribe in one click.