listmonk

System Architecture Diagram

Hover over each component to see its responsibilities:

Layered Architecture

listmonk follows a clean 4-layer architecture. Each layer has a single responsibility and communicates only with adjacent layers.

Layer 1: HTTP Handlers (cmd/*.go)

Thin handlers that parse HTTP requests (path params, query strings, JSON bodies), call the Core layer, and serialize responses. No business logic here. Each handler is a method on the App struct which holds references to all subsystems. Echo framework provides routing, middleware, and context.

Layer 2: Core Business Logic (internal/core/)

All domain operations: CRUD for subscribers, lists, campaigns, templates. The Core struct wraps the DB and query runner. This layer is pure Go with zero HTTP dependencies — it could be called from CLI, tests, or any other interface. It enforces validation, permission checks, and domain invariants.

Layer 3: Campaign Manager (internal/manager/)

The concurrent campaign processing engine. Runs as a long-lived goroutine that polls the DB for active campaigns. It owns the entire send pipeline: batch fetching, template rendering, rate limiting, worker pool dispatch, progress tracking, error handling. Completely rewritten in v3.0.0 for near-instant pause/cancel and lossless counting.

Layer 4: Data Layer (queries/*.sql + PostgreSQL)

All SQL lives in .sql files, loaded at startup via goyesql. No ORM. This gives full control over query optimization, PostgreSQL-specific features (JSONB, arrays, materialized views), and makes SQL reviewable and versionable. The sqlx library provides struct scanning.

Key Architectural Decision: The App Struct

type App struct {
    core       *core.Core           // Business logic layer
    fs         stuffbin.FileSystem   // Embedded filesystem (assets, SQL, i18n)
    db         *sqlx.DB              // PostgreSQL connection pool
    queries    *models.Queries       // Pre-loaded named SQL statements
    constants  *constants            // Runtime config snapshot
    manager    *manager.Manager      // Campaign processing engine
    importer   *subimporter.Importer // CSV/bulk import processor
    notifs     *notifs.Notifs        // Admin email notifications
    i18n       *i18n.I18n            // Internationalization
    bounceProc *bounce.Manager       // Bounce email processor
    captcha    *captcha.Captcha      // ALTCHA/hCaptcha
    auth       *auth.Auth            // Sessions + RBAC + OIDC
    events     *events.Events        // SSE event bus
    paginator  *paginator.Paginator  // Cursor/offset pagination
    log        *log.Logger
}

This is Go's idiomatic alternative to dependency injection containers. All subsystems are initialized in init.go and wired together via this struct. Handlers access them via a.core, a.manager, etc.

Database Schema (ER Diagram)

PostgreSQL schema with ~12 tables, JSONB for extensibility, materialized views for analytics:

Key Database Design Decisions

1. Custom ENUM Types (12 types)

PostgreSQL ENUMs enforce valid states at the DB level: campaign_status has 6 states (draft → running → scheduled → paused → cancelled → finished), subscriber_status has 3 (enabled, disabled, blocklisted), subscription_status tracks per-list relationship (unconfirmed, confirmed, unsubscribed). This is a state machine enforced by the database, not application code.

2. JSONB for Flexible Attributes

subscribers.attribs stores arbitrary key-value data as JSONB, enabling schema-less subscriber segmentation. You can query attribs->>'city' = 'Atlanta' with GIN indexes. The settings table stores all app config as JSONB key-value pairs, enabling runtime configuration changes through the UI without schema migrations.

3. Junction Table Pattern (subscriber_lists)

Many-to-many with a composite primary key PK(subscriber_id, list_id). The junction table carries its own state (subscription_status) and metadata (meta JSONB). Separate indexes on each FK column for efficient queries in both directions.

4. Keyset Pagination Columns

campaigns.last_subscriber_id and max_subscriber_id enable cursor-based pagination for campaign sending. Instead of OFFSET N (O(N) scan), it uses WHERE id > last_subscriber_id ORDER BY id LIMIT batch_size (O(1) via index). Critical for sending campaigns to millions of subscribers.

5. Materialized Views for Dashboard

Three materialized views precompute expensive aggregations: mat_dashboard_counts (subscriber/list/campaign totals), mat_dashboard_charts (30-day click/view trends), mat_list_subscriber_stats (per-list subscriber counts by status). Refreshed on cron schedule via REFRESH MATERIALIZED VIEW CONCURRENTLY. Orders-of-magnitude speedup for large databases.

6. Soft References & Denormalization

campaign_views.subscriber_id is nullable with ON DELETE SET NULL — if a subscriber is deleted, their view records persist for analytics. campaign_lists.list_name denormalizes the list name so campaign history survives list deletion. Preserves historical accuracy while allowing entity cleanup.

7. Indexing Strategy

Strategic indexes on: email (case-insensitive unique via LOWER(email)), status columns (for filtered scans), composite (id, status) for campaign batch fetching, DATE(created_at) expression indexes for time-series analytics. Partial unique index on templates(is_default) WHERE is_default = true ensures only one default template.

8. UUID + Serial ID Dual Identity

Internal operations use fast integer serial IDs for joins and pagination. External/public-facing operations use UUIDs (subscriber unsubscribe links, campaign archives, media references). Best of both worlds: performance internally, security externally (UUIDs aren't guessable).

Campaign Processing Pipeline

The campaign engine is the heart of listmonk. Rewritten in v3.0.0 for lossless operation:

Concurrency Model Deep Dive

Producer-Consumer with Channels

// Simplified mental model of the campaign engine

// Producer: fetches subscriber batches from DB
go func() {
    for {
        batch := db.Query("SELECT ... WHERE id > ? LIMIT ?",
                          lastSubID, batchSize)
        for _, sub := range batch {
            msg := renderTemplate(campaign, sub)
            msgChan <- msg  // Send to worker pool
        }
        lastSubID = batch[len(batch)-1].ID
        updateProgress(campaign, lastSubID)
    }
}()

// Consumer pool: N goroutines sending messages
for i := 0; i < concurrency; i++ {
    go func() {
        for msg := range msgChan {
            rateLimiter.Wait()          // Token bucket
            err := messenger.Push(msg)  // SMTP/HTTP
            if err != nil {
                handleRetry(msg, err)
            }
            atomic.AddInt64(&sent, 1)
        }
    }()
}

Rate Limiting Strategies

Crash Recovery & Resumption

The campaign stores last_subscriber_id after each batch. On restart, campaigns with status='running' are automatically resumed from the last checkpoint. The v3.0.0 rewrite ensures every single message is counted (not approximated), making pause/resume lossless. The to_send field is computed at campaign start as the total subscriber count for the target lists.

SMTP Connection Pool

Each configured SMTP server maintains a pool of max_conns persistent TCP connections. Connections are reused across messages (SMTP pipelining). idle_timeout and wait_timeout control connection lifecycle. Multiple SMTP servers can be load-balanced under the default "email" messenger, or targeted individually by naming them.

Go Patterns & Best Practices

// Messenger interface (Strategy pattern)
type Messenger interface {
    Name() string
    Push(msg Message) error
    Flush() error
    Close() error
}
// Implementations: email.Emailer, postback.Postback

type App struct {
    core    *core.Core
    manager *manager.Manager
    db      *sqlx.DB
    // ... all subsystems
}
// Methods: func (a *App) GetSubscribers(...)

-- queries/queries.sql
-- name: get-subscriber
SELECT * FROM subscribers WHERE id = $1;

-- name: get-campaign-subscribers
SELECT ... WHERE id > $1 ORDER BY id LIMIT $2;

internal/
  core/          # Business logic (unexportable)
  manager/       # Campaign engine (unexportable)
  bounce/        # Bounce processing
  auth/          # Authentication
  media/         # Media storage
  subimporter/   # CSV import

sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGHUP)

go func() {
    <-sigChan
    srv.Shutdown(ctx)     // HTTP
    manager.Close()       // Campaigns
    db.Close()            // Postgres
    syscall.Exec(...)     // Self-replace
}()

// Precedence (last wins):
// 1. CLI flags (--config, --install)
// 2. TOML file (config.toml)
// 3. Env vars (LISTMONK_*)
// 4. Database settings table

ko.Load(file.Provider("config.toml"),
        toml.Parser())
ko.Load(env.Provider("LISTMONK_", ...))

Additional Go Idioms Used

Scalability & Performance at 1M+ Scale

Proven Production Numbers

What Makes It Fast?

Bottlenecks & Scaling Limits

What Would You Do to Scale to 100M+ Subscribers?

Great system design interview follow-up:

What Happens When 1 Million Requests Hit listmonk?

listmonk handles 6 fundamentally different request types. Each follows a different hot path through the system. Understanding these flows is critical for system design interviews — an interviewer will ask "walk me through what happens when..." and expect you to trace from TCP accept to DB write to response.

Flow 1: Tracking Pixel Open (Highest Volume — 1M+ per campaign)

When a subscriber opens an email, their client loads a 1x1 transparent PNG. This is the hottest path in the system.

1. EMAIL CLIENT ─── GET /campaign/{campaignUUID}/{subscriberUUID}/px.png ───▶ ECHO ROUTER
      │
      │  // No auth middleware — public endpoint. No session lookup.
      ▼
2. ECHO ROUTER ─── matches route "/campaign/:campUUID/:subUUID/px.png" ───▶ HANDLER: handleCampaignPixel()
      │
      │  // Parse UUIDs from path params. Validate format (fast regex). No DB lookup yet.
      ▼
3. HANDLER ─── checks privacy.disable_tracking setting ───▶ DECISION POINT
      │
      ├── IF tracking disabled: Return 1x1 PNG immediately. No DB write. O(1).
      │
      ├── IF privacy.individual_tracking = false:
      │     Insert into campaign_views with subscriber_id = NULL (anonymous).
      │     // Still counts the view, but can't attribute to a subscriber.
      │
      └── IF individual tracking enabled:
            │
            ▼
4. DB INSERT ─── INSERT INTO campaign_views (campaign_id, subscriber_id, created_at) ───▶ POSTGRESQL
      │
      │  // campaign_id resolved from UUID via campaigns table lookup (indexed).
      │  // subscriber_id resolved from UUID via subscribers table lookup (indexed).
      │  // Two UUID→ID lookups + one INSERT. Total: 3 queries.
      │  // campaign_views has BIGSERIAL PK — write-optimized append-only table.
      ▼
5. RESPONSE ─── 200 OK, Content-Type: image/png, body: 1x1 transparent PNG (68 bytes) ───▶ EMAIL CLIENT

Flow 2: Link Click Tracking (High Volume — 100K+ per campaign)

Every link in a campaign email is wrapped. When a subscriber clicks, they hit listmonk first, which records the click then redirects to the actual URL.

1. BROWSER ─── GET /link/{linkUUID}/{campaignUUID}/{subscriberUUID} ───▶ ECHO ROUTER
      │
      │  // Public endpoint. No auth. Three UUIDs in path.
      ▼
2. HANDLER ─── handleLinkRedirect()
      │
      ├── Resolve link UUID → link record (get actual URL)
      │     // links table: url TEXT NOT NULL UNIQUE. UUID indexed.
      │
      ├── Resolve campaign UUID → campaign_id
      │
      ├── Resolve subscriber UUID → subscriber_id (if individual tracking on)
      │
      ├── INSERT INTO link_clicks (campaign_id, link_id, subscriber_id)
      │     // BIGSERIAL PK. Indexes on campaign_id, link_id, subscriber_id, DATE.
      │     // 4 index updates per INSERT. Heavier than campaign_views.
      │
      └── 302 REDIRECT → actual URL
            // User sees the destination page. Redirect is instant.

Flow 3: Campaign Send Pipeline (1M Outbound Emails)

This is not a request flow — it's a server-initiated push pipeline. But it's what people mean by "1M requests" in the context of listmonk.

MANAGER GOROUTINE (long-lived, polls DB every ~5s)
      │
      ├── SELECT campaigns WHERE status IN ('running','scheduled')
      │     // status column indexed. Cheap scan — usually 0-5 active campaigns.
      │
      ▼  For each active campaign:
BATCH PRODUCER (one goroutine per campaign)
      │
      │   // Loop until all subscribers processed:
      ├── SELECT subscribers WHERE id > last_subscriber_id
      │     AND id IN (subscriber_lists WHERE list_id IN campaign_lists)
      │     AND status = 'enabled'
      │     ORDER BY id LIMIT 1000  // ← keyset pagination, O(1) via PK index
      │
      │   // For each subscriber in batch:
      ├── TEMPLATE RENDER ─── Go html/template.Execute()
      │     // Inject: subscriber.name, subscriber.attribs, campaign.subject
      │     // Generate: tracking pixel URL, wrapped link URLs
      │     // Sprig functions available. Compiled template cached (sync.Once).
      │     // CPU cost: ~0.1ms per render for simple templates.
      │
      ├── msgChan <- message  // Send to buffered channel. Blocks if channel full (backpressure).
      │
      └── UPDATE campaigns SET last_subscriber_id = ?, sent = ?  // Checkpoint after batch

                          ║ channel (buffer = batch_size)
                          ▼
WORKER POOL (N = app.concurrency, default 10 goroutines)
      │
      │   // Each worker loops: for msg := range msgChan { ... }
      │
      ├── RATE LIMITER ─── rateLimiter.Wait()
      │     // Token bucket: app.message_rate tokens/sec
      │     // OR sliding window: app.message_sliding_window_rate per window_duration
      │     // Blocks goroutine until token available. Natural throttle.
      │
      ├── MESSENGER.Push(msg) ─── SMTP connection from pool
      │     // Acquires connection from pool (max_conns per SMTP server)
      │     // SMTP EHLO → MAIL FROM → RCPT TO → DATA → message body → QUIT
      │     // Connection reused for next message (persistent TCP). Pipelining.
      │     // On failure: retry up to max_msg_retries times
      │
      ├── atomic.AddInt64(&sent, 1)  // Lock-free counter update
      │
      └── ON ERROR: increment error counter. If errors > max_send_errors → PAUSE campaign

1M Emails — Time & Resource Breakdown

Flow 4: Public Subscription Form Submit

When a user subscribes via a public form on your website. Lower volume but has more steps.

1. BROWSER ─── POST /subscription/form ───▶ ECHO ROUTER
      │
      ├── CAPTCHA VERIFY ─── ALTCHA proof-of-work check OR hCaptcha API call
      │     // ALTCHA: CPU verification, no external API call (~1ms)
      │     // hCaptcha: external HTTP call to verify token (~100-300ms)
      │
      ├── DOMAIN FILTER ─── check email domain against blocklist/allowlist
      │     // privacy.domain_blocklist: ["*.disposable.com", "tempmail.org"]
      │     // In-memory check. O(N) against list but lists are tiny.
      │
      ├── UPSERT SUBSCRIBER ─── INSERT ... ON CONFLICT (email) DO UPDATE
      │     // Case-insensitive: idx_subs_email ON LOWER(email)
      │     // If exists: update name/attribs. If new: create with UUID.
      │
      ├── INSERT subscriber_lists ─── subscribe to requested lists
      │     // Status = 'unconfirmed' for double-optin lists
      │     // Status = 'confirmed' for single-optin lists
      │
      ├── IF double-optin: SEND CONFIRMATION EMAIL
      │     // Render optin template with confirmation link
      │     // Push to SMTP via messenger (same pool as campaigns)
      │     // Confirmation link: /subscription/optin/{subscriberUUID}/{listUUID}
      │
      └── 200 OK ─── render success template page

Flow 5: REST API Request (Admin/Programmatic)

API-triggered operations: creating subscribers, managing lists, triggering transactional emails. Authenticated path.

1. CLIENT ─── GET /api/subscribers?page=1&per_page=50 ───▶ ECHO ROUTER
      │
      │  // Authorization header: "username:api-token" (base64)
      ▼
2. MIDDLEWARE CHAIN
      │
      ├── Auth Middleware ─── validate API token
      │     // Lookup user by username in users table (indexed)
      │     // Verify token (bcrypt compare or direct match for API users)
      │     // Load user role + permissions from roles table
      │     // Set auth context on echo.Context
      │
      ├── Permission Check ─── does user have "subscribers:get_all" permission?
      │     // Check user_role permissions array. If list-scoped, filter by allowed lists.
      │
      └── CORS Middleware ─── check Origin header against security.cors_origins
      ▼
3. HANDLER ─── GetSubscribers()
      │
      ├── Parse query params ─── page, per_page, query, list_id, order_by
      │
      ├── Build SQL ─── dynamic WHERE clause from search/filter params
      │     // If "query" is plain text: ILIKE search on email and name
      │     // If "query" starts with SQL expression: parse as raw WHERE clause
      │     // Permission-filtered: only show subscribers in user's allowed lists
      │
      ├── EXECUTE SQL ─── via prepared statement from goyesql
      │     // Uses sqlx.Select() for struct scanning
      │     // COUNT(*) for total (separate query)
      │     // OFFSET/LIMIT pagination for API (not keyset — acceptable for admin UI)
      │
      └── 200 OK ─── JSON response with data[] + total + per_page + page

Flow 6: Bounce Webhook (SES/SendGrid/Postmark)

After a campaign, bounce notifications flow back from email providers. Volume scales with send volume — expect 2-5% bounce rate.

1. SES/SENDGRID ─── POST /webhooks/bounce/{type} ───▶ ECHO ROUTER
      │
      ├── Authenticate webhook ─── verify provider-specific auth
      │     // SES: verify SNS signature. SendGrid: verify key. Postmark: basic auth.
      │
      ├── Parse bounce payload ─── extract: email, bounce type, campaign ID
      │     // Normalize across providers into internal bounce_type ENUM
      │
      ├── INSERT INTO bounces ─── (subscriber_id, campaign_id, type, source, meta)
      │     // Subscriber looked up by email. campaign_id nullable (might be unknown).
      │
      ├── CHECK THRESHOLD ─── count bounces for this subscriber by type
      │     // SELECT COUNT(*) FROM bounces WHERE subscriber_id = ? AND type = ?
      │     // Compare against configured actions: hard.count=1, soft.count=2
      │
      └── IF threshold exceeded: UPDATE subscribers SET status = 'blocklisted'
            // Or DELETE subscriber if action = 'delete'
            // Cascading: subscriber_lists entries also cleaned up (FK CASCADE)

Flow 7: Transactional Email API

Single message sends triggered by your application (welcome emails, password resets, order confirmations). Different from campaign sends — synchronous, one-at-a-time.

1. YOUR APP ─── POST /api/tx ───▶ ECHO ROUTER
      │
      │  // Body: { "subscriber_email": "...", "template_id": 5, "data": {...} }
      │  // Auth: API token (required)
      ▼
2. HANDLER
      │
      ├── Resolve subscriber ─── lookup by email or ID
      ├── Load TX template ─── from template cache (in-memory)
      ├── Render template ─── with subscriber data + custom data payload
      ├── messenger.Push(msg) ─── synchronous SMTP send
      │     // Blocks until SMTP ACK or error. Caller waits.
      │     // No rate limiting — each TX call = one immediate send.
      └── 200 OK ─── { "data": true }

Request Flow Summary — All 7 Flows at 1M Scale

Will Goroutines Run Into Memory Errors?

Short answer: the campaign engine is safe, but the HTTP server is not protected. listmonk has two completely different goroutine models running simultaneously, and they have very different risk profiles.

Go Goroutine Memory Model — The Basics

The Two Goroutine Models in listmonk

// Fixed pool — goroutine count = concurrency (constant)
for i := 0; i < concurrency; i++ {
    go worker(msgChan)  // Exactly N goroutines. No more.
}
// Producer blocks if channel full — natural backpressure
msgChan <- msg  // Blocks here, NOT by spawning new goroutines

// Go's net/http — UNBOUNDED goroutine creation
func (srv *Server) Serve(l net.Listener) error {
    for {
        conn, _ := l.Accept()
        go srv.serve(conn)  // New goroutine for EVERY connection!
    }                        // No limit. No backpressure.
}

When Does OOM Actually Happen?

The key distinction is concurrent vs total requests. 1M requests over an hour is fine. 1M requests in 1 second is a problem.

The Real Danger: Goroutine Pile-Up on DB Pool

The OOM risk isn't from goroutine creation — it's from goroutine accumulation. Here's the cascade failure:

CASCADE FAILURE SCENARIO — 50K req/sec tracking pixel burst

t=0s    50,000 requests arrive. Go spawns 50,000 goroutines. (~3.5 GB)
          │
          ▼
t=0.001s All 50K goroutines try to acquire a DB connection from pool (max_open=25).
          25 goroutines get connections. 49,975 goroutines BLOCK waiting.
          │
          ▼
t=0.005s Each DB query takes ~5ms. First 25 connections freed. Next 25 goroutines proceed.
          But another 50K requests arrived! Now 99,950 goroutines blocked. (~7 GB)
          │
          ▼
t=1.0s   At 50K/sec inflow, 25 conn pool processes ~5000 req/sec (25 × 200 queries/sec).
          Deficit: 45,000 goroutines/sec accumulating. After 1 second: ~45K blocked.
          │
          ▼
t=10s    ~450K goroutines blocked. Memory: ~30 GB. GC thrashing. Latency spikes.
          │
          ▼
t=20s    ~900K goroutines. OOM killer triggers. Process killed.
          Campaign sending (which shares the same process) dies too.

What listmonk Does NOT Have (Protection Gaps)

How You'd Fix This — Production-Grade Architecture

PRODUCTION FIX: Bounded Concurrency + Async Writes

// 1. Add server-level connection limits
srv := &http.Server{
    Addr:         ":9000",
    ReadTimeout:  10 * time.Second,   // Prevent slow-read attacks
    WriteTimeout: 15 * time.Second,   // Prevent goroutine hang on slow clients
    IdleTimeout:  60 * time.Second,   // Close idle keep-alive connections
    MaxHeaderBytes: 1 << 20,          // 1MB header limit
}

// 2. Add concurrency limiter middleware
sem := make(chan struct{}, 10000)      // Max 10K concurrent requests
e.Use(func(next echo.HandlerFunc) echo.HandlerFunc {
    return func(c echo.Context) error {
        select {
        case sem <- struct{}{}:
            defer func() { <-sem }()
            return next(c)
        default:
            return c.String(503, "server busy")  // Load shedding
        }
    }
})

// 3. Async tracking writes (decouple from request path)
trackChan := make(chan TrackEvent, 100000)  // Buffered channel

// Handler: write to channel, return immediately
func handlePixel(c echo.Context) error {
    select {
    case trackChan <- TrackEvent{campID, subID}:
        // Queued successfully
    default:
        // Channel full — drop event (acceptable for analytics)
    }
    return c.Blob(200, "image/png", pixel1x1)  // Instant response
}

// Background flusher: batch inserts every 100ms
go func() {
    ticker := time.NewTicker(100 * time.Millisecond)
    batch := make([]TrackEvent, 0, 1000)
    for {
        select {
        case ev := <-trackChan:
            batch = append(batch, ev)
            if len(batch) >= 1000 {
                flushBatch(batch)        // COPY INTO campaign_views
                batch = batch[:0]
            }
        case <-ticker.C:
            if len(batch) > 0 {
                flushBatch(batch)
                batch = batch[:0]
            }
        }
    }
}()

// 4. DB call timeouts
ctx, cancel := context.WithTimeout(c.Request().Context(), 3*time.Second)
defer cancel()
db.QueryContext(ctx, query, args...)   // Cancels if DB slow

// 5. Monitor goroutine count
go func() {
    for range time.Tick(5 * time.Second) {
        n := runtime.NumGoroutine()
        metrics.Gauge("goroutines", n)  // Prometheus metric
        if n > 50000 {
            log.Error("goroutine count critical", "count", n)
        }
    }
}()

Interview Answer: "How Does Go Handle 1M Concurrent Requests?"

Contention & Concurrency — Where Things Fight Over Shared Resources

Concurrency is about structure (multiple things can run). Contention is about conflict (multiple things fighting for the same resource). listmonk has both, and understanding the contention points is what separates a senior answer from a junior one in interviews.

Resource Contention Map — Every Shared Resource in listmonk

DB Pool Contention — The Priority Inversion Problem

This is the most important contention point in listmonk and a great interview discussion topic:

PRIORITY INVERSION: All subsystems share 25 DB connections

┌─────────────────────────────────────────────────────────┐
│                  sql.DB Pool (max_open=25)               │
│   ┌────┐┌────┐┌────┐┌────┐┌────┐  ...  ┌────┐          │
│   │conn││conn││conn││conn││conn│       │conn│  ×25     │
│   └──┬─┘└──┬─┘└──┬─┘└──┬─┘└──┬─┘       └──┬─┘          │
└──────┼─────┼─────┼─────┼─────┼──────────────┼───────────┘
       │     │     │     │     │              │
       ▼     ▼     ▼     ▼     ▼              ▼
  Campaign Campaign Pixel  Pixel  API          Bounce
  Batch1   Batch2   Track  Track  List          Check

Problem scenarios:

1. Campaign sending large batches holds connections for batch SELECT (~5-50ms).
   Meanwhile, admin API requests queue behind campaign queries.
   Admin dashboard feels slow during active campaigns.

2. 1M tracking pixel INSERTs saturate the pool.
   Campaign batch fetch can't get a connection.
   Campaign throughput drops. Send time extends.

3. Materialized view REFRESH (cron job) takes 30-60 seconds.
   Holds 1 connection during entire refresh.
   24 connections left for everything else.

4. Bulk subscriber import doing 10K upserts.
   Competes with campaign sending for connections.
   Both slow down.

Row-Level Contention in PostgreSQL

Go-Level Concurrency Primitives Used

Concurrent Campaign Execution — Overlapping Lists

What happens when two campaigns target overlapping subscriber lists simultaneously?

SCENARIO: Campaign A targets List 1 (500K subs), Campaign B targets List 2 (500K subs)
          200K subscribers are on BOTH lists.

Campaign A goroutines:        Campaign B goroutines:
  1 batch producer               1 batch producer
  10 send workers                10 send workers
  ──────────────                 ──────────────
  11 goroutines                  11 goroutines    // Total: 22 goroutines

What happens to the 200K overlapping subscribers?

  ✓ Both campaigns independently fetch and send to them.
  ✓ The subscriber receives BOTH emails (intentional — different campaigns).
  ✓ No deduplication across campaigns (by design).
  ✓ No row locks conflict — campaigns only SELECT from subscriber_lists.
  ✓ Each campaign has independent last_subscriber_id cursor.
  ✓ Each campaign has independent sent counter (atomic).

Contention points:

  1. DB Pool: 22 goroutines competing for 25 connections.
     Batch fetches: 2 long-running SELECTs.
     Workers doing progress UPDATEs: occasional contention.
     Mitigation: workers mostly wait on SMTP (I/O bound), not DB.

  2. SMTP Pool: 20 workers sharing max_conns connections.
     If max_conns=10, workers queue for connections.
     Mitigation: each campaign can use different named SMTP servers.

  3. Rate Limiter: GLOBAL rate limit (message_rate) shared across campaigns.
     Two campaigns each wanting 100 msg/sec with rate=100 → each gets ~50.
     Campaign throughput halves with each concurrent campaign.

  4. No campaign-level resource isolation.
     A slow campaign (complex template) slows all campaigns
     by holding DB connections longer.

Race Conditions — Known & Potential

Interview Answer: "How Do You Handle Contention in a Concurrent System?"

Resilience & High Availability

Failure Modes & Recovery

What listmonk Does NOT Have (HA Gaps)

Data Integrity Guarantees

PostgreSQL provides ACID transactions for all subscriber/list/campaign mutations. FK constraints with CASCADE ensure referential integrity. ENUM types enforce valid state transitions. The sent counter in v3.0.0 is exact (not approximated), ensuring no duplicate or missed sends on pause/resume.

Idempotent Upgrades

./listmonk --upgrade is idempotent. Running it multiple times has no side effects. Migrations use version checks and are applied sequentially. Critical for automated deployment pipelines (Kubernetes rollouts, CI/CD).

Design Principles & Patterns

GoF & Architectural Patterns Catalog

Non-Functional Requirements — How listmonk Handles Them

NFRs are the make-or-break qualities that interviewers probe after your functional design. listmonk is an excellent case study because it's a production system handling millions of messages — every NFR decision below was battle-tested, not theoretical.

Security

Privacy & GDPR Compliance

Observability & Logging

Operability & Deployment

Performance (as NFR)

Internationalization (i18n)

Maintainability & Testability

Error Handling & Fault Tolerance

Go-Specific Scalability Patterns

Go Scalability Mental Model for Interviews: listmonk proves that a single Go process with goroutine pools + channels + connection pooling can handle millions of operations. The key insight: Go's concurrency model maps perfectly to the producer-consumer pattern. The producer is I/O-bound (DB fetch), workers are I/O-bound (SMTP send), and channels decouple them. You don't need Kafka for this workload — Go channels are an in-process message queue. You add Kafka when you need multi-node distribution or replay guarantees.

High Availability (HA) — What Exists & What Doesn't

┌──────────────┐     ┌──────────────────────────────────────────────┐
│   Load       │     │  Application Tier                            │
│   Balancer   │────▶│  ┌─────────────┐  ┌─────────────┐           │
│   (Nginx/    │     │  │ listmonk    │  │ listmonk    │           │
│    ALB)      │     │  │ (active     │  │ (passive    │           │
│              │     │  │  sender)    │  │  read-only) │  × N      │
└──────────────┘     │  └──────┬──────┘  └──────┬──────┘           │
                     └─────────┼────────────────┼──────────────────┘
                               │                │
                     ┌─────────▼────────────────▼──────────────────┐
                     │  Database Tier                               │
                     │  ┌──────────┐    ┌──────────┐               │
                     │  │ Postgres │───▶│ Postgres │               │
                     │  │ Primary  │    │ Replica  │  (streaming)  │
                     │  └──────────┘    └──────────┘               │
                     └─────────────────────────────────────────────┘

Active sender: processes campaigns, handles writes
Passive instances: serve UI/API reads, handle public pages
Primary DB: all writes, campaign state
Replica DB: passive instances read from here

NFR Summary Matrix — Interview Quick Reference

When an interviewer asks "how would you handle X?" — point to these concrete implementations:

The gaps are intentional trade-offs for simplicity. In an interview, acknowledge them and propose solutions: "listmonk prioritizes operational simplicity — a single binary serving 7M+ emails. At scale, I'd add: /healthz endpoint for K8s probes, Prometheus metrics via expvar or echo middleware, structured logging with slog (Go 1.21+), circuit breakers per SMTP server using sony/gobreaker, exponential backoff with jitter on retries, and a dead letter table for failed messages. For horizontal send scaling, I'd introduce Kafka between the batch producer and send workers, with campaign partition assignment via etcd distributed locks."

System Design Interview Cheat Sheet

Use listmonk as a reference when answering questions about designing email/notification systems, producer-consumer pipelines, or self-hosted SaaS alternatives.

If Asked: "Design a Newsletter/Email System"

Key Talking Points

Quick-Reference: Numbers to Know

Related System Design Problems

If you study listmonk deeply, you can answer variations of these interview questions: Design a notification system (email/SMS/push), Design a mailing list manager, Design a campaign analytics platform, Design a self-hosted SaaS tool, Design a producer-consumer pipeline with rate limiting, Design a system with crash recovery and exactly-once processing, How would you handle millions of concurrent email sends, Design a multi-tenant newsletter platform.