R2 — Atomicity across machines: distributed transactions and why you avoid them

R0 ended on a promise: atomicity does not survive the trip across the network. This section is what that costs, and what the industry does instead.

Where the boundary bites

Atomicity — all of it happens or none of it does — is something one database gives you inside one transaction. It does not stretch across two systems. Your database transaction cannot un-send a message the broker already accepted, and the broker cannot un-accept one when your transaction aborts.

There is a mechanism for spanning two systems — the distributed transaction — and the rest of this section is why you will not be using it.

So every time an operation touches something outside its own database — a broker, another service, a payment rail, an email provider — you are on the wrong side of this boundary and need a deliberate answer.

The distributed transaction

Everything in R3, R4 and R5 is a workaround for not having this. It only makes sense once you know what it replaces.

What a transaction gives you. ACID: atomicity (all or nothing), consistency (constraints hold), isolation (concurrent transactions do not see each other's partial work), durability (committed means survived). Inside one database this is solved, cheap and reliable. The moment an operation spans two databases, or a database and a broker, none of it extends across the gap.

What a distributed transaction is. One logical unit of work that changes data in more than one independent system — two databases, or a database and a message broker — with the same promise a local transaction makes: either every system commits its part, or none of them does.

The components of a distributed transaction

A distributed transaction always involves the same four roles. Naming each one precisely, rather than saying "the system", shows the interviewer you know where every responsibility sits — and therefore where every failure lands.

One absence in that table explains most of the modern objection: Kafka is not a resource manager. Nor are most HTTP services, payment rails or SaaS APIs. The protocol needs every participant to implement prepare-and-wait, and the ecosystem you actually build on largely does not.

How the components talk: the two-phase commit protocol

The components coordinate through two-phase commit (2PC), a protocol that splits "commit" into a vote and a decision:

1. Phase 1 — prepare. The transaction manager asks every resource manager whether it can guarantee to commit its part. A resource that says yes first writes its changes durably and keeps its locks; from then on it has given up the right to decide on its own.
2. The decision. If every vote is yes, the transaction manager writes COMMIT to its log; if any vote is no, it writes ABORT. The decision is that log write, not the messages that follow it.
3. Phase 2 — commit or roll back. The transaction manager tells every resource the outcome; each one applies it and releases its locks.

Step through the four scenarios below. The first is the happy path and the second shows a single "no" vetoing the whole transaction — both are 2PC working correctly. The third is the failure that disqualifies it: watch what the resource managers are still holding after the coordinator dies. The fourth shows what people do under pressure when it stays dead.

The protocol is correct. The problem is what happens when the coordinator dies between the two phases.

A participant that has voted yes has surrendered its right to decide. It must hold its locks and wait for an answer that may never come — it cannot commit (the other participant may have voted no) and cannot abort (the other may already have committed). This is the blocking window, and it is not a bug to be engineered away; it is a proven property of the protocol.

The interface they talk through: XA

Two-phase commit only works across products from different vendors if they agree on the exact calls. XA is that agreement: an open standard from X/Open (now The Open Group) defining the interface between a transaction manager and its resource managers. XA is the interface; two-phase commit is the protocol it carries.

In Java the same interface appears as JTA: every XA-capable driver implements javax.transaction.xa.XAResource (jakarta.transaction.xa in newer Jakarta EE), whose start, end, prepare, commit, rollback and recover map one-to-one onto the calls above. That is why a resource manager in the table above is precisely "a system that ships an XA driver" — and why Kafka, which does not, cannot take part.

Where 2PC sits: CAP, PACELC and ACID

This is where the theory earns its keep, because it shows the blocking window is not a flaw in the implementation — it is a choice the protocol makes on your behalf, every time.

CAP, stated properly. Consistency, availability, partition tolerance — and the common phrasing, "pick two of three", is misleading. In a distributed system partitions are not optional; the network will drop messages whether you planned for it or not. So the real statement is narrower: when a partition happens, you must choose between consistency and availability. Keep answering and risk disagreeing, or stop answering and stay correct.

PACELC: the trade-off on a normal day. CAP only tells you what happens during a partition — when machines cannot reach each other. Partitions are rare; almost all of the time the network is working fine, and CAP says nothing about that time. PACELC adds it. The name reads as a sentence: if there is a Partition, choose Availability or Consistency; Else — no partition, a normal day — choose Latency or Consistency.

That second half is the one you feel every day. Keeping copies in agreement means waiting for other machines to confirm each write, and that waiting is latency: a write that must be confirmed by a replica in another region pays a cross-region round trip, tens of milliseconds, on every write, even though nothing is broken. Answering from the nearest copy without waiting is fast, but may return slightly stale data.

What 2PC chooses, in both halves:

So 2PC is PC/EC: the consistency corner in both halves. Nothing about that is wrong — it is exactly what you want for a transfer between two accounts in one bank's ledger. It is simply the most expensive corner, and you are paying for it on every transaction, not only on the rare day a partition occurs.

ACID and 2PC. The point of 2PC is to stretch a single database's ACID guarantee across several independent resource managers. It is worth being precise about how much of it actually stretches:

• Atomicity — genuinely delivered, and this is the whole purpose. All resources commit or none do, decided by one log write.
• Isolation — delivered too, but by holding locks in every participant from prepare until commit. The isolation is real; its duration is now a function of the network.
• Consistency — never global. Each resource still enforces its own constraints; nothing checks an invariant that spans two of them. That gap is exactly what reconciliation (R6) exists to cover.
• Durability — comes from each resource's own log, plus the coordinator's. The coordinator's log is the one that makes recovery possible, which is why it is a component in its own right.

Two implications worth saying out loud. First, ACID is a property of a transaction, not of a system — so "we need ACID across microservices" is usually a sentence that has not been thought through, because the only honest way to get it is to make everyone wait. Second, the industry's answer is not to weaken correctness but to relocate it: keep strict ACID inside each service, and use compensation plus reconciliation to converge between them. That is what BASE names — basically available, soft state, eventual consistency — and it is a deliberate trade, not a lowering of standards.

Why this is disqualifying in practice

• Availability multiplies, it does not average. A transaction spanning three services each at 99.9% succeeds at about 99.7%, and it fails whenever any participant or the coordinator is down. You have coupled the availability of everything to the weakest member.
• Locks are held across the network. Lock duration becomes a function of someone else's latency and GC pauses. Under load this is where throughput dies.
• Most things you want to include do not support it. Kafka does not do XA with your database. Nor do most HTTP services, payment rails, or SaaS APIs. The protocol needs every participant to implement prepare-and-wait, and the modern ecosystem largely does not.
• It reintroduces a coordinator to fail over, with its own transaction log, its own recovery, its own operational burden.

Where it legitimately still lives. Inside a single database across shards, where one vendor controls every participant. In systems that hide it behind consensus rather than a bare coordinator — Spanner and CockroachDB run 2PC with Paxos-replicated participants, so a coordinator failure does not block indefinitely. Within Kafka, where transactions cover topics and offsets inside one cluster. And in legacy enterprise stacks with a real XA transaction manager, usually between two databases in the same data centre.

The alternatives, and what each gives up

Since distributed ACID inevitably couples the availability of every participant, the industry made a different bargain: keep real ACID inside each local database, and accept eventual consistency across the network. That is the philosophical shift. The three approaches below are not drop-in replacements for a distributed transaction — they are what you build once you have stopped asking for one.

The protocol approach, for completeness:

Eventual consistency paradigms — local ACID plus network-level convergence:

TCC is worth recognising by name, because it is 2PC's shape without its blocking: try reserves, confirm commits the reservation, cancel releases it. Each phase is an ordinary call that can be retried, nobody holds a database lock across the network, and the reservation is explicit domain state rather than an invisible lock. A funds hold is TCC.

TCC in practice: an e-commerce checkout paid by card

TCC is not only a textbook pattern. Every card payment is one: authorisation places a hold on the cardholder's funds (try), capture turns the hold into a charge (confirm), and a void — or simply letting the authorisation expire — releases it (cancel). Seat and ticket holds with a countdown, hotel reservations and a bank's funds earmarks have the same shape, and Apache Seata, the distributed-transaction framework Alibaba open-sourced, ships a TCC mode for services that want it as a framework. In most stacks, though, TCC lives in the domain model as explicit holds rather than in a framework.

An online checkout is the clearest worked example, because it has to move two things that live in different systems, stock and money, and must not move one without the other. Neither the inventory database nor the card network can join an XA transaction, but each of them offers a natural hold.

The flow follows from the table:

1. The order service writes the order as PENDING with its idempotency record (R3), then sends both tries.
2. Both tries succeed: it writes CONFIRMED, then confirms the stock straight away and captures the card when the goods ship. Capturing at dispatch is why a cancellation before shipping costs a void rather than a refund.
3. Either try fails — out of stock, or the card is declined: it writes CANCELLED, then cancels whichever tries did succeed.

The order service's state write is the decision — exactly like the TM log in 2PC — but nobody blocks while waiting for it, because every hold carries its own expiry.

The rules that make TCC correct

Two of these are obvious, two are not, and frameworks such as Seata exist largely to enforce the last two.

The last two are handled by one small record per transaction and participant, written in the same local transaction as the reservation:

An authorisation that expires before capture is a cancel you did not send. If shipping can take longer than the authorisation stays valid, the design needs a re-authorisation before capture, and reconciliation (R6) must catch any order marked confirmed whose capture later failed.

Step through the checkout below. The first tab is the happy path and the second a declined card. The third crashes the coordinator after both tries, so you can watch the holds expire on their own instead of blocking anyone. The fourth delivers a try late, after its cancel, to show the empty rollback and suspension rules.

Where TCC beats a plain saga is the undo. A saga's compensation is invented afterwards — a refund, a correcting entry — and can itself fail. TCC's cancel releases something that was reserved for the purpose of being released, so it is guaranteed to be possible. That is why R5 calls TCC a saga with a better first step: use it wherever the domain already has a hold.

Semantic locks. Notice what TCC's reservation actually is, and what a saga's PENDING status does. A database lock is invisible infrastructure — nobody outside the engine knows it exists. When you set a row to PENDING, or place a hold on ₹5,000, you are doing the same job in the domain model instead: a flag that other operations are expected to respect. That is a semantic lock — isolation re-implemented at the application layer, because the database can no longer provide it across services.

Two consequences follow, and both are worth saying out loud. It only works if every code path honours it, since nothing enforces it for you. And it is visible to users, which is why a bank shows "available balance" separately from "current balance" — the semantic lock has leaked into the product, deliberately.

What to say in the interview

When asked "how do you handle distributed transactions?", refuse the premise, then show the ladder: "I avoid them — 2PC couples availability and blocks on coordinator failure. I keep the atomic part inside one database using an outbox, use TCC — explicit reservations with an expiry, like a card authorisation and a stock hold at checkout — where the domain has a hold, use sagas with compensation where state spans owners, and reconcile against anything outside my control."