HLD · Transactions
Saga Pattern & Distributed Transactions
Coordinate multi-service workflows without a single global 2PC lock.
1. Why not 2PC (two-phase commit)?
In classic two-phase commit (2PC), a coordinator asks every participant to prepare and then decides whether all should commit or abort. Participants may hold resources while waiting, and coordinator or network failures can block progress. This is often a poor fit for long-running workflows across independent services.
Interview: "We use sagas for long-running business processes across services."
2. What is a saga?
A sequence of local transactions. Each step commits in its own service. If a step fails, run compensating transactions to undo prior steps.
3. Example: travel booking
- Book flight — Flight service reserves seat
- Book hotel — Hotel service reserves room
- Charge card — Payment service captures payment
If step 3 fails → compensate: cancel hotel, cancel flight (async refund events).
3b. State table (orchestrated saga)
| Step | State | If success | If fail → compensate |
|---|---|---|---|
| 1 | FLIGHT_BOOKED | Book hotel | Cancel flight |
| 2 | HOTEL_BOOKED | Charge card | Cancel hotel → cancel flight |
| 3 | PAID | Done | Refund → cancel hotel → cancel flight |
User sees PENDING until step 3 completes — not half-confirmed bookings.
4. Choreography vs orchestration
| Choreography | Orchestration |
|---|---|
| Services react to events (no central brain) | Central saga orchestrator drives steps |
| Loose coupling; harder to visualize flow | Clear state machine; single place for logic |
| Good for simple flows | Good for complex sagas (Temporal, Camunda) |
Choreography example: OrderCreated event → Payment listens → PaymentFailed event → Inventory listens and releases stock.
5. Compensating transactions
Not always literal undo — semantic reverse:
- Reserve seat → release seat
- Charge $50 → refund $50 (idempotent refund_id)
- Send email → send cancellation email (can't unsend)
Compensations must be idempotent, meaning they are safe to retry. They can also fail, so the workflow must record state and retry or alert an operator.
6. Isolation challenges
Between saga steps, other users see intermediate state (hotel booked but not paid). Mitigations:
- Pending state: Mark booking "PENDING_PAYMENT"
- Semantic lock: Hold inventory with TTL until payment completes
7. Outbox pattern (reliable events)
DB write + event publish atomically: write business row and outbox row in same transaction; separate poller publishes to Kafka.
Prevents "order saved but event lost" race.
Concrete example: The Order service inserts order 123 and an OrderCreated outbox row in one database transaction. If Kafka is down, the order remains saved and the poller publishes the pending outbox event after Kafka recovers.
8. Tools
Temporal, AWS Step Functions, Cadence — durable workflow engines with retries, timers, compensation handlers.
Quick revision
- Saga: Local TX steps + compensations on failure.
- Not 2PC: Better availability; eventual consistency across services.
- Choreography = events; Orchestration = central coordinator.
- Compensations must be idempotent.
- Recovery: Persist saga state because compensating actions can also fail.
- Outbox for reliable event publishing with DB write.