A payment can commit successfully while a slow Amazon SES response turns the same request into an error screen. A transactional email queue architecture keeps receipts and password resets off that request path while giving each message a recoverable route to a provider.
A queue stores a durable job after the business event commits; a worker then renders and submits the message. The worker applies rate limits, retries temporary failures with bounded backoff, and records outcomes so duplicate jobs and exhausted attempts can be handled deliberately.
For a small SaaS, the practical starting point is often an outbox table in the existing database plus one worker. Add RabbitMQ or another broker when you have a specific need for broker routing or independent queue scaling. Neither the queue nor a successful provider submission proves that an email reached an inbox.
Below, we follow a subscription receipt from a database transaction to a provider response, show a usable outbox and claim query, and compare a home-grown job table with RabbitMQ and a sending service. The goal is not to add infrastructure for its own sake; it is to make a user-facing action reliable even when email processing is delayed.
Why transactional email queue architecture needs a queue
When an application sends a message inline, the request that handles a signup, payment, or password reset waits for the sending provider. A slow response stretches request time; a timeout can return an error even if the database transaction or payment has already succeeded. That mismatch is particularly confusing when a user sees a failure screen after an account has been created or a payment recorded.
A queue separates the business result from the notification attempt. The application commits the business change and a durable email job, then returns its response. A worker processes the job independently, which means a provider delay can grow the email backlog without blocking every signup or payment request.
This pattern is useful for messages tied to a specific event: account verification, password resets, payment receipts, order updates, and security alerts. It does not make delivery instantaneous, and it does not turn an external provider into a reliable dependency. It gives the application a place to retain work, apply policy, and recover when the provider is temporarily unavailable.
For example, a subscription service can commit a payment record and a receipt job in one database transaction. The response confirms the payment, while the queue tracks whether the receipt is pending, in progress, accepted by the provider, or awaiting investigation. If the worker is paused for ten minutes during an incident, the user’s payment request does not also have to wait ten minutes.
The separation is valuable at low volume too. One application process and one background worker can provide it; the design does not require microservices, a cluster of brokers, or a dedicated email operations team. Start with a clear boundary between business success and email processing, then add operational components only when the workload calls for them.
Reference architecture: from application event to delivery
A reliable path separates the event that creates a message from the worker that sends it. For a password reset, the application validates the request, creates a reset token, stores the token record, and writes a job referencing that record. It can return its HTTP response without waiting for the provider.
The path is business event → durable job → worker → provider API or SMTP relay → provider response → application send record. Delivery events, when available through the provider and configured for the account, should be processed as a separate stream; do not label an accepted send request as delivered. This distinction matters because the send worker and the later outcome processor answer different questions.
Put a durable handoff next to the business transaction
If the database update and queue publish happen in separate systems, a failure between them can leave a committed payment without a receipt job. A transactional outbox reduces that gap: the application writes the business record and an outbox row in the same database transaction, and a publisher later claims rows and forwards them to the job queue.
A minimal PostgreSQL schema might look like this:
CREATE TABLE email_outbox (
id bigserial PRIMARY KEY,
event_key text NOT NULL UNIQUE,
message_type text NOT NULL,
recipient text NOT NULL,
template_key text NOT NULL,
payload jsonb NOT NULL DEFAULT '{}',
state text NOT NULL DEFAULT 'pending',
attempts integer NOT NULL DEFAULT 0,
available_at timestamptz NOT NULL DEFAULT now(),
locked_at timestamptz,
last_error text,
provider_ref text,
created_at timestamptz NOT NULL DEFAULT now(),
sent_at timestamptz
);
CREATE INDEX email_outbox_ready_idx
ON email_outbox (available_at, id)
WHERE state IN ('pending', 'retry');
The business transaction inserts the email row with a stable event_key, such as receipt:invoice_8472. The unique constraint prevents the same logical event from creating multiple outbox rows if a payment webhook or application command is replayed. Keep the payload limited to data the worker needs to find the right message; store sensitive tokens and mutable account details in the application database rather than copying them into a long-lived queue payload.
A PostgreSQL worker can claim a small batch without making two workers process the same row at once:
WITH claimed AS (
SELECT id
FROM email_outbox
WHERE state IN ('pending', 'retry')
AND available_at <= now()
ORDER BY available_at, id
FOR UPDATE SKIP LOCKED
LIMIT 20
)
UPDATE email_outbox AS job
SET state = 'processing', locked_at = now(), attempts = attempts + 1
FROM claimed
WHERE job.id = claimed.id
RETURNING job.*;
Commit the claim before making the network request. After the send attempt, write the provider result and next state. Add a lease-recovery process for rows stuck in processing after a worker crash: once a lock is older than the maximum expected send duration plus a safety margin, return the job to retry or mark it for review. The chosen timeout depends on the provider request timeout and your workload, so measure it rather than copying an arbitrary production value.
Keep the queue payload distinct from the message template
A job should identify the recipient, message type, locale, template key, business record, and stable event ID. The worker loads authoritative data and renders the template at send time. That makes it possible to correct a template without storing a second copy of every rendered HTML message in the queue.
Avoid putting reset tokens, payment details, or API credentials into broker messages unless your security and retention model explicitly allows it. A short-lived reference to a database record is usually easier to protect and audit. Record the rendered template version or a content hash if you need to explain exactly which content was submitted later.
The worker should record distinct states such as pending, processing, retry, accepted, failed_permanent, and dead_letter. Use accepted to mean the provider accepted the submission, not that the recipient saw it. The exact later outcome depends on the provider’s event facilities and your event-processing setup.
Choose a queue that fits your scale
Use the simplest job store that preserves work during the failures you expect and gives your team enough control over scheduling and retries. A database-backed queue works well when the app already runs PostgreSQL and its email volume does not compete materially with core queries. A dedicated broker such as RabbitMQ becomes useful when broker routing, separate scaling, or database contention is a concrete constraint.
| Option | What it provides | Operational trade-off | Good fit |
|---|---|---|---|
| Database-backed queue or outbox | Jobs share the application database’s transaction and durability model; a worker claims rows and updates job state. | The team owns claiming, leases, retry scheduling, cleanup, and database capacity. | One application, a small number of message types, and a workload the database can serve. |
| RabbitMQ | Durable queues can recover metadata after a node restart; persistent messages routed to durable queues are persisted before publisher confirmation. | The team operates a broker, configures routing and acknowledgements, and monitors recovery and queue growth. | Workloads that need broker routing or queue processing scaled separately from database reads and writes. |
| Sending provider such as Amazon SES | Accepts send requests through its API or SMTP interface and applies account sending quotas and rates. | A sending interface is not a substitute for an application-owned durable queue or job-state model. | The application retains jobs and wants SES to handle the sending step. |
| Volanea sending endpoint or SMTP relay | Volanea’s endpoint processes suppression checks, contact upserts, template rendering, tracking, and delivery in one pipeline; an SMTP relay is also available for transactional sending. | The application still needs to decide how it stores and retries work before submission. | Teams that want a managed sending pipeline while keeping their own enqueueing and recovery policy. |
RabbitMQ durability has two separate parts. A durable queue preserves queue metadata, while messages must also be published as persistent; for quorum queues, publisher confirmation means a quorum accepted and confirmed the message. Publisher confirms cover the publisher-to-broker side, while consumer acknowledgements cover processing, so neither mechanism alone proves that an email was sent. See RabbitMQ’s queue durability documentation and publisher confirms guide.
There are useful choices before operating a broker yourself. For example, pg-boss and Graphile Worker use PostgreSQL-backed job processing; BullMQ uses Redis; and RabbitMQ is a dedicated message broker. These libraries have different job semantics and operating requirements, so choose based on the infrastructure your team already knows and the recovery behavior the application needs.
Amazon SES is a sending service, not the application’s job table. Its quotas are per AWS Region and count recipients rather than messages: a message to 10 recipients counts as 10 against the quota. SES’s sending rate is the maximum it can accept each second, and it may accept mail more slowly than that limit. Keep your own durable queue in front of SES when you need application-controlled scheduling, retry policy, and visibility into pending work (SES sending limits).
Volanea also fits at the send step, not as a reason to omit your application’s durable handoff. Its sending endpoint processes suppression checks, contact upserts, template rendering, tracking, and delivery in one pipeline. We do not treat that endpoint or SES as a replacement for the outbox and retry state described here; the queue’s job is to preserve the application event until a send can be attempted.
Design jobs for retries, idempotency, and failure recovery
A send job may run more than once. Workers restart, networks time out, brokers redeliver, and a process can fail between the provider request and the queue acknowledgement. Design for bounded retries and idempotent handling instead of assuming exactly-once execution across the application, broker, and provider.
Use a bounded schedule for temporary failures
For a general transient failure, one concrete policy is to retry after 5 seconds, 30 seconds, 2 minutes, 10 minutes, and 30 minutes, adding random jitter of up to 20% to each delay. Cap attempts at five and then move the job to a dead-letter state. This is an example application policy, not a universal provider rule: the appropriate schedule depends on the message’s urgency and the provider’s specific response.
Classify errors before scheduling a retry. A network timeout or a temporary throttling response may clear without a code change. A malformed address or invalid template is not fixed by waiting, so mark it as a permanent failure and send it to investigation or correction rather than repeatedly submitting the same bad data.
Amazon SES has distinct advice for its own limit errors. For API or SDK sends that exceed an account sending limit, SES returns a ThrottlingException with Daily message quota exceeded or Maximum sending rate exceeded; AWS recommends waiting for an interval of up to 10 minutes before retrying. For SMTP sends above an account limit, the client might receive 454 Throttling failure: Maximum sending rate exceeded or 454 Throttling failure: Daily message quota exceeded. SES SMTP troubleshooting recommends progressively longer waits for 400 errors, for example 5 seconds, 10 seconds, then 30 seconds; after the third retry fails, wait 20 minutes and repeat. A 500 error requires correcting the problem before resubmitting. Use the response path that matches your API or SMTP integration: SES API quota errors and SMTP troubleshooting.
Make the logical notification idempotent
Use a stable key for the business event, not a new key for each attempt. For example, receipt:invoice_8472 identifies one receipt regardless of how often the worker retries it. Store the key under a unique database constraint, and check the send record before issuing another provider request.
A basic worker guard might be:
async def process(job, db, provider):
async with db.transaction():
record = await db.fetchrow(
"SELECT state FROM email_outbox WHERE event_key = $1 FOR UPDATE",
job.event_key,
)
if record is None or record["state"] in ("accepted", "dead_letter"):
return
# Mark the attempt before leaving the transaction.
await db.execute(
"UPDATE email_outbox SET state = 'processing' WHERE event_key = $1",
job.event_key,
)
result = await provider.send(job)
await record_provider_result(db, job.event_key, result)
This prevents concurrent workers from casually sending the same job, but a database guard cannot make an external send and a database commit atomic. If the provider accepts a message and the worker crashes before saving the response, the job’s true outcome is uncertain. If the provider supports an idempotency mechanism, pass the same stable key on every retry; otherwise, record the ambiguity and use provider logs or later events to reconcile before deciding whether to resend.
Acknowledge only after processing, and retain exhausted jobs
With RabbitMQ manual acknowledgements, acknowledge after the worker has recorded the send result. If a worker dies before acknowledging, RabbitMQ can requeue the message; the consumer must be able to handle a redelivery. RabbitMQ recommends idempotent consumers for this reason, and its redelivered flag is only a hint that a delivery may have been seen before, not proof of prior processing (reliability guidance).
Move jobs that exhaust their retry budget into a dead-letter queue (DLQ) or equivalent failed-job store. Preserve the original event key, recipient reference, attempt history, timestamps, last provider response, and error classification. Provide an operator action to inspect and replay a job after correction; replay should preserve its logical idempotency key so that a previously accepted message is not automatically sent again.
For a database queue, the failed-job table or a dead_letter state can serve the same operational purpose as a broker DLQ. Keep a reason and a replay history rather than deleting the row on failure. The team should be able to answer whether the message was never attempted, rejected, accepted with an uncertain local result, or deliberately suppressed.
For a fuller treatment of send retries and when not to retry, see our guide to transactional email retry logic.
Control throughput and protect deliverability
Set worker concurrency against the provider’s actual quota and rate, not the maximum speed of the queue. Amazon SES sending quotas are separate for each AWS Region, and its sending quota is based on recipients. Its sending rate is a per-second acceptance limit that may be exceeded for short bursts but not sustained periods; SES may accept messages more slowly than the stated maximum (SES sending limits).
Use a rate limiter shared by workers that send through the same SES account and Region. If four workers each send at a locally configured rate of 10 messages per second, their combined rate can exceed an account limit of 10 per second. Apply the rate limit to recipients, not just job count, when one job can include multiple recipients. When SES responds with throttling, slow the shared limiter and schedule retries instead of adding more workers to a provider-side bottleneck.
Separate operational messages from campaigns or other lower-priority mail. A password reset should not wait behind a large campaign batch, so assign time-sensitive jobs a priority lane or separate queue and reserve capacity for them. Volanea provides campaigns, event-triggered automations, and segments alongside transactional sending, but the application should still consider which traffic needs independent priority and rate controls (Volanea).
Process bounces and complaints as recipient and reputation signals, not merely as dashboard counts. Update suppression state so future jobs do not keep sending to addresses that should no longer receive messages. Monitor operational and campaign traffic separately where possible: a campaign spike in negative recipient responses can affect the reputation of a sending domain also used for account notices. There is no single complaint threshold that applies to every provider, domain, and recipient mix.
Authentication is supporting infrastructure for the sending domain, not a substitute for queue controls. RFC 7208 defines SPF as a way for a domain’s administrators to authorize hosts to use that domain in SMTP MAIL FROM or HELO identities. An SPF record is one string in a DNS TXT record for the relevant name; multiple SPF records at the same owner name are not permitted. The RFC’s example is v=spf1 +mx a:colo.example.com/28 -all. SPF evaluation also has a limit of 10 DNS-querying terms, including include, a, and mx, so adding mechanisms without counting lookups can break evaluation.
DKIM uses a signing domain and selector to identify a DNS TXT key lookup. For example, selector foo.bar and signing domain example.com produce foo.bar._domainkey.example.com (RFC 6376). A successful signature check says that the signed content has not changed since signing; it does not prove end-to-end message integrity or, by itself, authenticate the visible author. DMARC evaluates alignment between the author domain and a validated SPF or DKIM identity, so configure all three with a clear understanding of which domain each one authenticates (RFC 9989).
Monitor queue health and delivery outcomes
A useful dashboard combines the state of work waiting to be sent with what happened after a send attempt. Queue metrics show whether jobs are being processed; provider and recipient outcomes show whether submissions are being accepted and what happens afterward. Keep those states separate so a low queue depth does not get mistaken for successful delivery.
Track at least these signals by message type and provider, and by AWS Region for SES:
- Queue depth and oldest-job age: Count pending work and measure how long the oldest job has waited. Age is often more actionable than total count because 500 jobs may be normal for one service and urgent for another.
- Enqueue-to-start delay: Measure time from the business event to a worker starting the job. This shows whether the queue itself is creating a delay.
- Worker duration and provider response time: Separate template rendering and database time from the send request. A worker slowdown and a provider slowdown need different responses.
- Retry rate and final failures: Graph attempts per message, exhausted retries, and error categories. A sudden rise in throttling should be visible before the backlog becomes large.
- Provider outcomes: Record accepted requests, provider errors, and later bounce or complaint signals where available. Treat an accepted request and a later recipient outcome as distinct records.
- Suppression and duplicate prevention: Count jobs skipped because an address is suppressed and jobs stopped by the idempotency guard. A sudden change can reveal a bad import, replay, or application bug.
Set alert thresholds from a delivery expectation and your observed baseline. For a service that expects password-reset requests to enter processing within a minute, an alert at a five-minute oldest-job age is a reasonable starting rule; it is an operational choice, not an SES guarantee. For example, alert on more than 1% throttled attempts over a rolling 10-minute window, then tune that threshold after you see normal traffic and provider responses. Pair percentage alerts with a minimum sample count so one failed request at low volume does not page the team.
Use at least two alert paths. One should detect a growing queue or old job; another should detect a provider error or throttle spike. If age rises while worker capacity is low and provider responses remain normal, investigate the worker pool or database claim query. If age and throttling rise together, inspect SES rate controls and account quotas before increasing concurrency. If jobs are accepted but recipient outcomes deteriorate, investigate bounce, complaint, authentication, and suppression signals instead of scaling the queue.
Make a dashboard traceable to an individual event. A send record should connect the business event key to the template, attempt history, provider response, and later outcome when available. Volanea provides send records, delivery, open, click, bounce, and unsubscribe statistics, plus event logs; these are useful application-side visibility, but the queue should still preserve its own pending, retry, and failure state. This distinction lets an engineer tell whether a message is waiting, submitted, or has a reported downstream outcome.
A practical implementation sequence for a SaaS team
Add asynchronous handling to one existing workflow before redesigning the application around messaging. A payment receipt is a good starting example because it has a clear source event and a durable invoice record. Keep the payment logic in the application, and add a job handoff that can be tested independently.
-
List message types and delivery expectations. Record the triggering event, recipient, template, priority, and acceptable processing delay. Separate password resets and receipts from campaigns or low-priority account summaries so that a large batch cannot quietly consume all worker capacity.
-
Write the job in the business transaction. Insert the invoice and its receipt outbox row in the same database transaction. Give the event a unique key such as
receipt:invoice_8472; a repeated webhook then hits the uniqueness constraint instead of creating another logical receipt. -
Run one worker end to end. Have it claim a row, load the account and template data, send through the selected provider, and record the result. Amazon SES supports sending through its API, including an AWS SDK, and through SMTP (SES sending methods). A managed sending endpoint can simplify suppression, rendering, and tracking at the send step, but the application still owns the durable job and its retry rules.
-
Set rate controls before adding concurrency. Find the sending quota and rate for the specific SES Region and account. In the SES sandbox, an account can send up to 200 emails per 24-hour period and 1 email per second; it can send only to verified email addresses and domains or the SES mailbox simulator (sandbox access). Outside the sandbox, the 24-hour quota and rate vary by use case. Pace jobs against the account’s actual limits, counting recipients rather than message jobs.
-
Add bounded recovery and an operator path. Implement error classification, retry delays, stale-lock recovery, a DLQ or failed-job state, and an idempotency check. Give an operator enough information to tell whether a job was rejected, accepted, or left uncertain after a worker interruption.
-
Instrument before migrating more mail. Put queue age, provider errors, retries, and outcomes on a dashboard. Exercise a throttle, a temporary provider timeout, a permanent message error, and a worker crash after submission but before recording the result. Confirm that each case produces the intended retry, terminal state, or investigation path.
-
Expand one message type at a time. Move signup messages and password resets after the receipt path behaves correctly. Keep a rollback path to the previous sender for a limited period, but do not let both paths send the same business event; use one event key and a single owner for each message type during migration.
Volanea can serve as the sending step in this design through its API or SMTP relay. For SMTP, the login username is volanea and the password is the project’s secret API key. A sk_test_ key renders and logs sends but never delivers them, which makes it suitable for exercising the worker and template path without sending production mail. Its sending endpoint handles suppression checks, contact upserts, template rendering, tracking, and delivery in one pipeline; keep your queue’s job of preserving and scheduling the business event distinct from that endpoint’s send processing.
Plan capacity around the actual sending limit as well as the provider’s own constraints. Volanea offers 1,000 free emails per month, and when a project reaches its monthly limit, sending pauses rather than incurring overage billing. Test the queue’s behavior at that boundary: hold jobs for later review or alert operators, rather than assuming a provider will continue sending beyond the project limit. Volanea’s email platform brings campaign and automation functions alongside the sending pipeline, but your application should continue to prioritize and account for critical messages explicitly.
A queue is reliable when it preserves the meaning of the business event as well as the message itself. A receipt should be traceable to one invoice, a retry should not create a second logical notification, and a provider incident should show up as a queue or response signal before users report missing mail. That is achievable with an outbox, a worker, bounded retries, and clear operational measurements before a team needs more infrastructure.
FAQ
How do I decide when to replace a database queue with RabbitMQ?
Move when you can point to a real constraint: database contention from job polling, a need for broker routing, or queue processing that must scale separately from application queries. A PostgreSQL outbox remains appropriate when its load is manageable and its transaction boundary is valuable. RabbitMQ adds durability and routing tools, but also a broker to operate and monitor.
What should happen when the sending account reaches its monthly limit?
Stop or hold new sends and alert the team; do not let the worker retry continuously against a limit that will not clear through a short backoff. Volanea pauses sending at the monthly limit and does not bill overages. Keep the pending jobs visible so the application can decide whether to resume, cancel, or prioritize them when sending is available again.
Can the worker guarantee exactly-once email delivery?
Not across a database, queue, network, and provider unless the whole path offers coordinated idempotency. A worker may receive an ambiguous timeout after the provider accepted a message. Use a stable event key, prevent concurrent duplicate jobs, and reconcile uncertain sends before replaying; that reduces duplicates without claiming an impossible guarantee.
Should campaigns and password resets share one queue?
They can share infrastructure, but time-sensitive operational messages should not sit behind a large campaign batch. Use separate priority lanes or queues and reserve processing capacity for resets, receipts, and security alerts. Volanea supports campaigns alongside transactional sending, so the application’s priority policy remains important even when both types use one sending platform.
Start free with 1,000 emails a month and test your queue’s send path before moving production traffic.