1. Situation
The cart API from the previous study now accepts checkout requests.
After an order is accepted, the business wants to:
- reserve inventory;
- authorize payment;
- arrange fulfillment;
- email or text the customer;
- update analytics; and
- notify fraud and operations systems.
Only one outcome must complete before the checkout API responds:
The order must be durably accepted—saved in storage that survives a request or process failure—or the client must receive a clear failure.
Analytics can be late. Email can be retried. A temporary shipping outage should not prevent new orders from being accepted. Payment and inventory need more careful coordination because one may succeed while the other fails.
This creates a new architecture problem:
synchronous acceptance -> asynchronous fulfillment and side effects
The services overlap at the edges, but they do not have the same central job.
2. Naive Design
Naive option 1: keep the entire order inside one request
The code is easy to trace when everything works. In production, every dependency becomes part of checkout availability and latency.
If analytics pauses for thirty seconds, the customer waits. If email fails, the system must decide whether to reject an otherwise valid order. If the client retries after a timeout, the entire chain may run again.
Naive option 2: publish an event before saving the order
Consumers receive OrderCreated, but the order database write then fails. They try to process an order that does not exist.
Naive option 3: save the order, then publish separately
1. database write succeeds
2. process crashes
3. event publish never happens
The code is trying to complete two separate writes: save the order in the database and publish an event to EventBridge. These writes go to different systems, so they are not one all-or-nothing operation. If the database write succeeds and the process crashes before publication, the customer sees an accepted order but fulfillment never hears about it. This failure window is the dual-write gap.
Naive option 4: use one SQS queue for every subscriber
Payment, analytics, and email workers all poll the same queue. Each message is normally processed by one competing consumer, so the services divide the messages instead of each receiving a copy.
A work queue is not automatically publish/subscribe (pub/sub) fanout, where one message is copied to every interested subscriber.
3. What Breaks
Follow four failure stories.
Failure 1: shipping is unavailable
In the synchronous chain, checkout fails even though the company could safely accept the order and arrange shipping later.
The system needs a durable handoff—usually a queue or event path—so shipping work can wait and retry without blocking order intake.
Failure 2: a payment response is lost
The payment provider authorizes the charge, but the worker times out before recording success. The message returns and is processed again.
A queue may deliver a message again when it cannot confirm the first processing attempt. AWS calls this at-least-once delivery. An idempotency key, such as the payment operation ID, is only an identifier; it prevents a second charge only when the payment path enforces it. For example, pass the same operation ID to a provider that records idempotency keys, so a retry returns the first authorization result instead of creating another charge.
Failure 3: one malformed message fails every batch
A Lambda function retrieves, or polls, ten SQS messages at once. This group is called a batch. Nine succeed and one malformed message fails. If the integration treats the whole batch as failed, all ten become visible again and repeat work.
The worker should report only the failed message so successful messages in the same batch do not repeat. A message that keeps failing should move to a dead-letter queue (DLQ) for investigation.
Failure 4: the queue keeps growing quietly
SQS preserves the work while a downstream service slows down. That prevents immediate loss, but it also hides a widening business delay unless the team monitors the age of the oldest message.
A queue is a buffer, not a cure. Limit consumer concurrency—the number of messages processed at the same time—to a rate the downstream service can sustain. Alarm when the oldest message shows that the business is falling behind.
4. AWS Architecture
Build the system by communication job.
Step 1: create a durable order-acceptance boundary
The checkout API has two promises to keep: save the order and make sure the rest of the system can eventually learn that the order was created. Here, OrderCreated is an event—a small message that records a business fact that has already happened.
The order database and EventBridge are separate systems. Ordinary application code cannot safely make their writes succeed or fail together. A transactional outbox solves this by first storing the event-to-be-published in the same database as the order.
The outbox is simply a database table or collection of messages waiting to be sent—similar to the Outbox folder in an email application. For order-8472, the flow is:
- Write the order record and an outbox record such as
OrderCreated: order-8472, pendingin one database transaction. - The database commits both records or neither record.
- After that commit, the API can safely return the accepted order ID.
- A separate publisher, often called an outbox relay, reads the pending record and publishes
OrderCreatedto EventBridge. - After publication succeeds, the relay marks the outbox record as sent or removes it, depending on the design.
If the application crashes after the database commit but before publication, the outbox record remains and the relay can try again. If the relay publishes the event and then crashes before marking the record as sent, it may publish the same event again. The outbox closes the missing-event gap; it does not create exactly-once delivery. Consumers must still be idempotent so processing the duplicate does not repeat a payment, reservation, or other business effect.
It is called transactional because the order and outbox records share one database transaction. EventBridge publication happens later and is not part of that transaction.
Another option is a reliable database change stream. The database exposes committed changes, and a publisher turns those changes into events. In either design, the goal is the same: preserve evidence of the committed order before relying on a separate publish call. Review the Outbox Pattern for deeper implementation detail.
For the rest of the lesson, assume OrderCreated describes an order that was successfully saved. It is a committed business fact, not a hopeful message sent before storage succeeds.
Step 2: use EventBridge to route the business fact
EventBridge is the router for business events. The producer sends OrderCreated to an event bus, which is the receiving channel inside EventBridge. A rule checks fields in each event, such as its source and type. When an event matches the rule, EventBridge sends a copy to the configured target—the destination, such as an SQS queue or a Step Functions workflow.
EventBridge is useful when producers should not know every consumer and routing depends on event content.
If EventBridge cannot deliver an event to a target, it can retry. Configure an EventBridge dead-letter queue when the event must be retained after those delivery attempts run out.
That DLQ answers:
Could EventBridge deliver the event to its configured target?
It is different from a consumer queue's DLQ. The EventBridge DLQ stores events the router could not deliver to a target. A consumer DLQ stores messages that reached the work queue but repeatedly failed while the consumer tried to process them.
Step 3: put SQS in front of independent workers that need buffering
For a consumer such as analytics, EventBridge and SQS solve two different parts of the delivery path:
OrderCreated -> EventBridge rule -> Analytics SQS queue -> Analytics worker
EventBridge decides that the event belongs to analytics. SQS then holds the analytics copy until its worker is ready. Combining them gives the system content-based routing first and a durable, consumer-controlled backlog second. EventBridge alone does not give the worker its own queue, while one shared SQS queue would make different consumers compete for messages instead of each receiving a copy.
Each queue receives the events for its consumer. Workers for the same queue compete to process its messages; separate queues let independent services each receive and retain their own copy.
SQS provides a buffer between receiving work and processing it. Consumers can run only as fast as the downstream system can safely handle; excess work waits durably in the queue. This provides backpressure control: a traffic spike becomes a growing backlog instead of an unlimited burst of parallel requests against a payment provider, database, or partner API.
Step 4: process SQS messages with retry safety
SQS holds the work; Lambda runs the code that processes it. AWS can connect them without the team running its own polling server: the Lambda service polls the queue, groups messages into a batch, and invokes the function with that batch. AWS calls this configured connection an event source mapping.
When a worker receives a message, SQS hides it from other workers for the visibility timeout. The message has not been deleted yet—it is temporarily checked out. If processing succeeds, the worker deletes it. If the worker crashes or does not delete it before the timeout expires, the message becomes visible and can be delivered again. That retry behavior is why the consumer must be idempotent.
Therefore:
- set the visibility timeout to cover expected processing and retry behavior;
- make the consumer idempotent;
- use partial batch responses so successful records are not retried merely because another record failed; and
- send repeatedly failing messages to a DLQ with enough context for diagnosis and replay.
The DLQ is an isolation area, not automatic resolution. Someone or something must inspect and fix the messages, then redrive them—move them back to a queue for another processing attempt—deliberately.
Step 5: use Step Functions when the business process has ordered state
Fulfillment is not a set of unrelated background jobs. The steps depend on one another: reserve inventory, authorize payment, and then arrange fulfillment. If payment fails after inventory was reserved, the system must decide what to do next—usually release that inventory.
Step Functions keeps track of the current step and chooses the next path after success or failure. It does not reserve inventory or authorize payment itself; each step invokes a Lambda function, an AWS service, or another API that performs the work. This combination separates workflow decisions from application logic.
The workflow definition is called a state machine. It describes the ordered steps, decision branches, timeouts, retries, and Catch paths, while the execution history shows what happened during a particular order.
The Release inventory branch is a compensating action. A distributed workflow cannot rewind the earlier reservation as though it never happened. Instead, it performs a new business operation that attempts to offset the earlier success—and that new operation can also fail or require a retry.
Standard Workflows keep durable execution history and fit long-running or auditable processes. Express Workflows are optimized for short, high-volume processes and have different execution and history guarantees. In either type, retries against payment, inventory, or other external systems still need idempotency.
Step 6: use SNS to copy one notification to several subscribers
After an order is confirmed, email, analytics, and operations may each need the same notification. Instead of making the producer send three separate messages, it publishes OrderConfirmed once to an SNS topic. SNS copies the message to every subscribed destination. AWS calls this one-to-many delivery fanout.
When those destinations are worker-based services, give each consumer its own SQS queue and subscribe those queues to the SNS topic:
OrderConfirmed
-> SNS topic
-> Email queue -> Email worker
-> Analytics queue -> Analytics worker
-> Operations queue -> Operations worker
SNS is responsible for making the copies. Each SQS queue safely holds one consumer's copy until that consumer is ready. If the email worker is slow or unavailable, its message waits in the email queue without blocking analytics or operations.
SNS alone provides fanout, but it does not give every worker its own durable backlog. One SQS queue alone provides a backlog, but its workers normally compete for each message. SNS + SQS provides both fanout and durable, consumer-specific buffering. SNS can also push directly to supported endpoints such as email or SMS when a separate worker queue is not required.
If EventBridge rules already send the event to every required application target, adding SNS may be redundant. Choose SNS when the main need is a straightforward broadcast to a set of subscribers or supported notification endpoints. Choose EventBridge when the main need is to inspect event content and route different event types to different targets.
Completed architecture
This is not a requirement to use every service in every workflow. Combine services only when their jobs are different: EventBridge chooses destinations, SQS holds each consumer's work, Lambda processes it, Step Functions coordinates dependent steps, and SNS broadcasts copies.
5. Request Or Data Flow
Learn three lifecycles.
Lifecycle 1: order acceptance
- The client sends a checkout request with an idempotency key.
- The API validates the request and records the order durably.
- In the same database transaction, the API records that
OrderCreatedstill needs to be published. - The API returns an accepted order ID.
- A relay publishes the committed fact to EventBridge.
The client no longer waits for analytics, email, or the entire fulfillment workflow.
Lifecycle 2: independent queued consumer
- An EventBridge rule sends
OrderCreatedto the analytics SQS queue. - Lambda polls a batch and processes each message.
- The handler uses the operation ID in one atomic conditional state change, or performs an update that is already idempotent.
- Successful messages are removed.
- Failed messages become visible for retry; repeated failures move to the consumer DLQ.
Lifecycle 3: coordinated fulfillment failure
- Step Functions reserves inventory.
- Payment authorization fails after its configured retry policy.
- A
Catchpath records the failure and requests inventory release. - The order state becomes failed or requires review.
- Operations can inspect execution history and business records.
The workflow makes the failure path visible. It does not make compensation infallible; compensating actions also need monitoring and retry safety.
6. Security Controls
Limit who can publish and consume
Give producers permission to publish only the expected events to the intended event bus. Give EventBridge the target permissions it needs. Give consumers access only to their queues, data stores, APIs, and secrets.
A resource policy is a permissions document attached to the receiving AWS resource. Use these policies carefully when an event bus, SNS topic, or SQS queue accepts access from another AWS account.
Keep sensitive data out of broad events
Events should normally contain identifiers and the safe context consumers need. Do not broadcast card details, secrets, authentication tokens, or unnecessary personal data through every target.
A useful event might say:
{
"detail-type": "OrderCreated",
"detail": {
"orderId": "order-8472",
"customerId": "customer-19"
}
}
Consumers retrieve protected details through authorized systems when necessary.
Encrypt and audit deliberately
Use service encryption and customer-managed AWS Key Management Service (KMS) keys when the organization needs direct control over an encryption key. Ensure the key policy and AWS Identity and Access Management (IAM) permissions allow the publishing and consuming services that need access.
Use AWS CloudTrail to audit configuration and AWS API changes. Use Amazon CloudWatch Logs for application and workflow behavior. Redact sensitive payloads before logging.
7. Resilience Controls
The order is not safely accepted until its durable state and publish intent survive the request. An event bus cannot repair an order that was acknowledged only in memory.
Assume messages and events can be delivered more than once. Build idempotent consumers for payments, inventory, notifications, and analytics. For local state changes, use stable operation identifiers and atomic conditional state transitions—one write that changes a record only when its current state matches the expected value—so two duplicate workers cannot both claim the same business step.
For SQS consumers:
- configure visibility timeout around processing behavior;
- configure a redrive policy, which moves messages to a consumer DLQ after repeated failed receives;
- enable partial batch failure reporting where appropriate; and
- monitor queue age, not only queue depth.
For EventBridge targets, configure retry and an EventBridge DLQ when delivery failures must be retained. Know which DLQ you are investigating.
For Step Functions, define Retry only for errors that are safe to retry, use Catch for explicit failure paths, and monitor failed or timed-out executions. Compensation should be represented as business state, not assumed to be a perfect rollback.
8. Performance Controls
SQS absorbs bursts but increases asynchronous completion time when arrival rate exceeds processing rate. That is controlled degradation, not free throughput.
Set Lambda concurrency so consumers drain the queue without overwhelming payment providers, databases, or partner APIs. The correct target is sustainable downstream throughput, not the fastest possible empty queue.
Batching reduces polling and invocation overhead but increases the amount of work affected by one invocation. Partial batch responses reduce unnecessary replay of successful records.
Choose queue type from ordering requirements:
- Standard queues favor high throughput and at-least-once delivery with best-effort ordering.
- First-In, First-Out (FIFO) queues preserve message order within their documented model and provide deduplication features, with different throughput and design constraints.
Use Step Functions Standard for durable, auditable coordination and Express for suitable short, high-volume workflows. Choose from execution requirements, not because one name sounds faster.
9. Cost Controls
Event-driven architecture replaces idle servers with several separately billed kinds of usage:
- EventBridge events and target invocations;
- SQS API requests and payload transfer;
- Lambda invocations, duration, and logs;
- Step Functions state transitions or Express execution resources;
- SNS deliveries;
- KMS, data transfer, and downstream service usage.
Filter events before invoking expensive consumers. Give EventBridge rules precise patterns and avoid accidental loops where a target emits an event that matches the same rule repeatedly.
Keep event payloads small. Store large documents in S3 or another suitable data store and send references plus integrity and identity metadata.
Batch SQS messages where it improves efficiency, but account for failure isolation and processing time. Retain logs long enough for operations without storing every payload indefinitely.
Do not introduce EventBridge, SNS, SQS, and Step Functions into the same branch unless each has a distinct requirement. Managed services reduce operations, not architectural cost or cognitive load.
10. Exam Variants
| If the question says... | Think... | Why |
|---|---|---|
| "Route events from many producers by content" | EventBridge | Rules match event fields and send events to targets |
| "Buffer work while a consumer is unavailable" | SQS | The queue retains backlog for polling workers |
| "Only one worker should process each work item" | SQS with competing consumers | A queue distributes work rather than broadcasting it to every worker |
| "Several subscribers need the same notification" | SNS topic, often with one SQS queue per consumer | SNS copies the message; each queue safely holds its consumer's copy until it is processed |
| "Coordinate ordered steps, retries, branches, and audit history" | Step Functions | A state machine makes workflow control explicit |
| "A message failed repeatedly and needs isolation" | SQS DLQ | Repeatedly failing messages leave the main consumer path for investigation |
| "EventBridge cannot deliver to its target" | EventBridge target retry and DLQ | This is a routing-delivery failure, not a consumer-processing failure |
| "Prevent duplicate payment after retry" | Idempotent consumer and stable operation ID | At-least-once delivery can repeat a message |
| "Preserve ordering for a message group" | SQS FIFO | FIFO is an ordering signal when its constraints fit |
| "Database commit must not lose event publication" | Transactional outbox or reliable change stream | It closes the dual-write gap |
11. Common Traps
| Trap | Better reasoning |
|---|---|
| "Every post-order action belongs in the checkout request." | Acknowledge after durable acceptance; move independently retryable work off the client path. |
| "Write the order, then publish, and assume both happen." | Preserve publish intent atomically with business state or derive it from a reliable change stream. |
| "EventBridge is just a durable work queue." | EventBridge routes events; SQS provides consumer-controlled backlog and polling. |
| "One SQS queue fans each message to every service." | Consumers on one queue compete; give independent subscribers separate queues or use fanout. |
| "SNS and SQS are interchangeable." | SNS makes one-to-many copies; SQS holds one consumer's work. Combine them when every consumer needs its own durable copy. |
| "A DLQ fixes bad messages." | A DLQ isolates failures; diagnosis, correction, and redrive still require an owner. |
| "Exactly-once workflow means payment needs no idempotency." | Retries and external side effects still require business-level duplicate protection. |
| "A growing queue means the system is resilient." | The work is retained, but increasing message age means the business is falling behind. |
| "Retry the entire Lambda batch when one record fails." | Use partial batch reporting when supported so successful records do not repeat unnecessarily. |
| "Compensation rolls the distributed system backward." | Compensation is a new business action that can also fail and need retries. |
Final Mental Model: One-Minute Review
| Term | Exact job | Remember it as |
|---|---|---|
| Transactional outbox | Save the order and its "event to publish" together; a relay publishes the event later | The database mail tray |
| EventBridge | Match business events and route them to selected targets | The mailroom sorting desk |
| SQS | Retain backlog and let workers process at a controlled rate | A durable inbox |
| Lambda consumer | Poll and process queued work | The inbox worker |
| Visibility timeout | Temporarily hide a received message while processing is attempted | The checked-out work slip |
| DLQ | Isolate repeatedly undeliverable or unprocessable work for investigation | The exception tray |
| Step Functions | Coordinate ordered steps, branches, retries, and failure paths | The workflow supervisor |
| SNS, optionally with subscriber SQS queues | Copy one notification to many subscribers; queues let each consumer process its copy independently | The announcement system with separate inboxes |
| Idempotency | Make repeated delivery produce one intended business effect | The duplicate stamp checker |
First, the transactional outbox makes a committed order discoverable even if the publisher crashes. After that, the central distinction is route versus buffer versus coordinate versus broadcast. EventBridge chooses destinations, SQS holds work, Lambda performs work, Step Functions manages workflow state, and SNS copies notifications to subscribers. Idempotency makes the design safe when a message is delivered more than once.
12. Related Topics
Use Amazon EventBridge for event buses, patterns, rules, targets, retries, and archives. Use Amazon SQS for queue types, visibility, redrive, and polling. Review AWS Step Functions for Standard and Express workflows, and Amazon SNS for topic fanout and notification endpoints.
Continue to Secure Partner File Ingest On S3 to apply the same buffering, duplicate-handling, and trust-boundary thinking to external file arrivals.
Official AWS references: