R19 — Reservations and contended inventory: BookMyShow, Amazon, Uber

R18 ended with a broker blocking a customer's money before sending an order: take it now, keep it if the order fills, give it back if it doesn't. That is a reservation, and it is the subject of this section. Here the scarce thing is a cinema seat, a unit of stock or a driver, and thousands of people may want the same one at the same moment.

The format is the same as R18's: the shared problem and its toolkit first, then three walkthroughs, each adding one difficulty to the last.

The shared concern — many people, one thing

Every system in this section has to guarantee one sentence: the same thing is never sold to two people — while still selling as much as possible, as fast as possible, and never keeping people waiting on something they cannot have. Five problems stand between you and that sentence. Each grows out of the one before.

Problem 1 — the race

The obvious code has a bug:

seat = SELECT status FROM show_seat WHERE seat = 'C4'     -- both users read AVAILABLE
if seat.status == 'AVAILABLE':
    UPDATE show_seat SET status = 'BOOKED' WHERE seat = 'C4'   -- both users write

Two requests can both read "available" before either writes, and both believe they won. This is the read-check-write race, and adding more servers makes it more likely, not less.

Problem 2 — put the check inside the write

The fix is to let the database decide the race in a single statement:

UPDATE show_seat SET status = 'HELD', hold_id = :me
 WHERE seat = 'C4' AND status = 'AVAILABLE';
-- 1 row updated: you won.   0 rows: someone else did.

The condition is checked while the row is locked, so exactly one request can pass it. The same idea appears as a compare-and-set (R8), a unique constraint, or an atomic decrement such as stock = stock - 1 WHERE stock >= 1. Whichever form you use, the rule is the same: never decide in application code what the database can decide atomically.

Problem 3 — holds that expire

People do not buy instantly. They pick seats, then spend a minute or two paying. If you mark the seat sold when they click, abandoned clicks lock seats forever; if you wait until payment succeeds, two people can pay for one seat. So the claim has two phases:

1. Hold — the item is taken off sale for a short time, say ten minutes.
2. Confirm or release — payment succeeds and the hold becomes a sale, or it fails, or time runs out, and the item goes back on sale.

The expiry is the safety net: if every other part of the system fails, the hold simply times out and the item returns. This is R2's try–confirm–cancel: try is the hold, confirm is the sale, cancel is the release.

Problem 4 — a payment in the middle

Between hold and confirm sits a payment provider you do not control. Its answers can be slow, duplicated or lost. So the flow is a saga (R5) with idempotency keys at every step (R3):

• the hold must outlast the payment window, or a customer can pay for a seat that has just been released;
• a payment callback can arrive twice, so confirming must be idempotent;
• a payment timeout is an unknown outcome (R0) — ask the provider, do not guess;
• if money arrives for a hold that already expired, refund it automatically.

Problem 5 — everyone wants the same item

Problems 2 to 4 are correct at any load. But when a blockbuster opens for booking, or a sale drops a thousand discounted TVs, a million people hit a few rows at once. Correctness holds — the conditional update still lets only one win — but the rows become queues and the database can fall over. So you put cheap layers in front that turn away the many before they reach the source of truth: a virtual waiting room, fast gates in Redis, splitting a hot row into buckets, or a single writer per hot item (R18's one-owner idea again). The fast layers may reject a buyer by mistake; only the database may say yes.

Rules of thumb for contended inventory

• Decide every race in one atomic statement, and read the row count to learn who won.
• Hold, then confirm or release — and give every hold an expiry.
• Make the hold outlast the payment window; refund anything that arrives late.
• Make confirm, release and payment callbacks idempotent.
• Shed load in front of the database, but let only the database say yes.

How the three walkthroughs build on each other

What is being reserved changes from one walkthrough to the next, and that drives the differences:

They are ordered so that each adds one difficulty:

1. BookMyShow — the purest reservation problem. Every seat is a distinct row, so you meet the race, the all-or-nothing hold of several seats, expiry, a payment in the middle and the booking-open stampede, with nothing else in the way.
2. Amazon — the units become interchangeable, so the claim becomes a decrement of a count, and the checkout becomes a longer saga across inventory, orders and payment. Most of the system — catalog, search, cart — is deliberately not strict, and the walkthrough shows where loose data has to become strict.
3. Uber — the scarce thing moves and must be found before it can be claimed. A hold lasts seconds, not minutes, and finding the right driver — a geospatial search over 375,000 location updates a second — is as hard as claiming them.

Two of these come back to money directly. Amazon's checkout is an R5 saga over R3 idempotency keys, and Uber charges once per trip through an external provider, with R3's key and R4's outbox in front of the call. R27, in the payments part of the pack, describes the payment platform both of them hand off to.

At a glance

Walkthrough 1 — BookMyShow, seats at the moment booking opens

Where this starts: the purest reservation problem. Every seat is a distinct row, so the race, the all-or-nothing hold for a group, expiry, the payment in the middle and the booking-open stampede all appear with nothing else in the way.

Scope and requirements

Interviewer: Design BookMyShow — people pick a film, a show and their seats, and pay.

You: The heart of it is selling each seat exactly once, even when fifty thousand people want the same evening show the minute booking opens. Before designing, a few questions:

Interviewer: Users pick exact seats. Seats are held for ten minutes while paying. Some large cinema chains run their own ticketing systems; most cinemas use ours. Payments through a PSP. Expect a few blockbuster openings a year where a couple of million people arrive in the first minutes.

You: Requirements, then.

Functional:

• Browse films, cinemas and showtimes by city.
• See a show's seat map — which seats are free, held or sold — and pick seats.
• Hold the chosen seats for ten minutes, all or none.
• Pay; on success the hold becomes a booking with a ticket and QR code; on failure or expiry the seats return.
• Cancel within the cinema's rules, with a refund.

Non-functional:

• Never sell a seat twice. No exceptions, including when the payment provider misbehaves.
• Never take money without either issuing a ticket or refunding it.
• Seat map loads in under 300 ms at p99; a hold returns in under 500 ms at p99.
• Survive a booking-open stampede without falling over, and stay fair while doing it.
• The seat map may be a few seconds stale; the hold must never be.

The seat map may be wrong for a moment; the hold must be right every time. That one sentence decides where we spend effort: cheap, cached reads for the map, and a strict, atomic write for the hold.

What's being probed: whether you find the one correctness rule and separate it from everything that can be approximate.

Concept — stale read, strict write. Most screens show slightly old data, and that is fine as long as the action they lead to is checked against the truth. A user who sees a seat as free that was taken a second ago just gets "sorry, that seat was just taken" and picks another. R19's introduction calls it putting the check inside the write: the map is advice, the hold is the decision.

Takeaway: Find the one thing that must never be wrong, make that write strict, and let the reads around it be cheap.

Capacity

Interviewer: Size it.

You: Assumptions for a normal day: 1M bookings, 2.5 seats each, about 20 seat-map views per booking, 20% of bookings in the evening peak hour. For a blockbuster opening: 2M users in the first ten minutes, on shows that hold about 500,000 seats in total.

Confirmed bookings can never exceed the number of seats, so the write path is small even at the peak. What explodes is reads — half a million a second of people looking — and collisions: thousands of people trying to hold the same few hundred seats of one show.

Storage is small: 10,000 screens × 5 shows a day × 300 seats = 15M seat rows a day. With two weeks of shows on sale that is about 200M rows of roughly 100 bytes — around 20 GB, easily partitioned by show.

What's being probed: whether you notice that writes are bounded by supply.

Concept — supply-bounded writes. In reservation systems, successful writes can never exceed the inventory. Whatever the demand, the database only has to accept as many holds as there are seats; everything else is rejection or waiting. So scaling the database is not the problem. Rejecting the excess cheaply — and protecting the hot shows from a pile-up — is. Amazon's flash sale, in the next walkthrough, has the same shape.

Takeaway: Size the read path for demand and the write path for supply, then design to reject the gap cheaply.

API

Interviewer: Sketch the API.

You: REST through the gateway, with an Idempotency-Key on everything that changes seats or money.

POST /v1/holds
Idempotency-Key: 9a7c2e14-0b6d-4f3a-8e21-6d5c4b3a2f10
{ "showId": "sh_8812", "seats": ["C4", "C5"] }

HTTP/1.1 201 Created
{ "holdId": "h_01J9ZQ", "seats": ["C4", "C5"], "amount": { "value": 64000, "currency": "INR" },
  "expiresAt": "2026-10-09T13:40:00Z" }

HTTP/1.1 409 Conflict
{ "error": "SEATS_UNAVAILABLE", "unavailable": ["C5"] }

The hold is all or nothing: two friends do not want one seat each in different rows. A 409 names the seats that were lost so the app can repaint them and let the user choose again. Money is an integer in paise. POST /v1/bookings returns 202 Accepted, because the PSP decides in its own time; the app follows the status.

Data model

Interviewer: Data model.

You:

All the seats of one show live on one partition, so holding several seats is a single local transaction — never a distributed one. A show is the natural unit: nobody ever holds seats across two shows at once.

High-level design

Interviewer: Draw it.

You: Browsing is a caching problem; holding is a correctness problem; paying is a saga. Play the flows in order: hold and book, a collision, an expired hold, then the booking-open stampede.

A booking, end to end:

1. The app loads the seat map from the seat-map cache — fast, possibly a second old.
2. The user picks C4 and C5. The seat service runs one conditional update on the show's partition; both rows move to HELD with a hold id and an expiry ten minutes out.
3. The user pays. The booking service starts a payment with the PSP, using the hold id as part of the idempotency key.
4. The PSP confirms. In one transaction the booking service turns the seats BOOKED, writes the booking, and puts BookingConfirmed in the outbox. The ticket goes out from the event.

Deep dive — the race for a seat

Interviewer: Two people tap C4 in the same millisecond.

You: Both send a hold. Both requests reach the same partition, and both run this statement:

UPDATE show_seat
   SET status = 'HELD', hold_id = :hold, hold_expires_at = now() + interval '10 minutes',
       version = version + 1
 WHERE show_id = :show AND seat_id IN ('C4', 'C5')
   AND (status = 'AVAILABLE' OR (status = 'HELD' AND hold_expires_at < now()));
-- rows updated must equal the number of seats asked for; otherwise ROLLBACK

The first to lock row C4 passes the condition and updates it. The second waits for the lock, re-checks the condition — C4 is now HELD and not expired — and updates nothing. It learns it lost from the row count: it asked for two seats and got fewer, so it rolls back and returns 409 with the seats it missed. No seat can be held twice, and a group is never split.

Step through it below: a hold and booking, two people racing for one seat, a group that cannot all be seated, an abandoned hold, a payment that arrives too late, and a booking-open queue.

Interviewer: Why not lock seats in Redis — SET seat:C4 me NX EX 600?

You: It is fast, and it is a fine gate in front of the database. But it is not safe as the truth. Redis replicates asynchronously, so if the primary fails just after granting a lock, the replica may never have heard of it and can grant the same seat again. R8 covers why a lock service without fencing cannot be the last word. And holding several seats atomically in Redis needs a script that takes all or none. The database's conditional update gives atomicity, durability and the all-or-nothing group in one statement, on a partition that only handles one show's few hundred rows.

What's being probed: an atomic multi-row claim, and knowing who wins without a lock service.

Concept — the row count is the verdict. A conditional update tells you how many rows met the condition. Ask for N seats, get N back: you won all of them. Get fewer: you lost at least one, and rolling back the transaction undoes the ones you did get. That gives all-or-nothing with no locks held across requests and no separate lock service. It is R19's introduction, problem 2, applied to several rows at once.

Takeaway: Claim every seat in one conditional update, compare the row count, and roll back on any shortfall.

Deep dive — holds that expire

Interviewer: A user holds seats and walks away.

You: Expiry is built into the hold statement itself: a seat whose hold has expired counts as free — note the hold_expires_at < now() in the condition. So the next person to try simply takes it. Correctness does not depend on any background job.

A sweeper still runs every few seconds, setting expired holds back to AVAILABLE — but only so the seat map stops showing them as held. If the sweeper stops, the map looks a little worse, but no seat is lost or sold twice.

UPDATE show_seat SET status = 'AVAILABLE', hold_id = NULL, hold_expires_at = NULL
 WHERE (show_id, seat_id) IN (
   SELECT show_id, seat_id FROM show_seat
    WHERE status = 'HELD' AND hold_expires_at < now() - interval '5 seconds'
    LIMIT 500
    FOR UPDATE SKIP LOCKED);

Interviewer: The PSP takes eleven minutes to confirm a payment, and the hold has expired.

A payment can succeed after its hold has expired — and someone else may now have the seat

You: Two defences, in order:

1. Make it rare. When the user starts paying, extend the hold once, say to fifteen minutes, and give the PSP a shorter timeout than that. The hold should outlive every normal payment.
2. Make it safe when it happens. Confirming is conditional on the hold still owning the seats: UPDATE show_seat SET status = 'BOOKED' WHERE hold_id = :hold AND status = 'HELD'. If the row count is short, the seats are gone, so the booking fails and the money is refunded automatically, with an apology and a discount code. Never "fix" it by bumping the other customer.

What's being probed: whether your design depends on a timer for correctness, and how you handle the late-success race.

Concept — lazy expiry. Instead of relying on a job to free expired holds, make "expired" part of the definition of "free". Then correctness needs no clock-driven process, and the background job only tidies up. The same trick appears in R17's leases and in caches with a TTL. The late payment is the reverse race: the outcome arrives after the claim lapsed, so the confirm must be conditional too — and the fallback is a refund, which is always possible, rather than a double sale, which is not.

Takeaway: Treat expired holds as free inside the claim itself, and make the confirm conditional so a late success becomes a refund.

Deep dive — the payment in the middle

Interviewer: Walk me through payment failures.

You: The booking is a saga with try–confirm–cancel steps (R2, R5):

The cases that catch people out:

• The PSP calls back twice. The callback handler confirms with WHERE hold_id = :hold AND status = 'HELD' and a unique holdId on the booking, so the second callback changes nothing and returns the same booking.
• The PSP times out. The outcome is unknown, not failed (R0). The payment service asks the PSP for the status of that idempotency key. Meanwhile the hold protects the seats, so there is no rush to guess.
• The user retries "Pay". Same idempotency key, so the PSP returns the first attempt's result instead of charging again.
• Nothing answers at all. The nightly reconciliation against the PSP's settlement file (R6) finds charges with no booking and refunds them.

Every failure ends in one of two states: a confirmed booking, or no booking and no charge. The hold's expiry and the automatic refund are what guarantee the second.

Deep dive — the booking-open stampede

Interviewer: A blockbuster opens for advance booking at noon. Two million people are waiting.

You: Correctness still holds — the conditional update cannot oversell. What breaks is everything around it: the seat-map cache gets hammered, popular shows' partitions turn into lock queues, and retries pile on. Four layers, each turning away load before it reaches the next:

1. A virtual waiting room. Before noon everyone lands on a waiting page. At noon they are admitted in random order — not first-come, which rewards bots — at roughly the rate the hold path can serve. Each admitted user gets a short-lived signed token that the gateway checks on every seat request.
2. Cheap, pushed seat maps. Instead of half a million refreshes a second, admitted users subscribe to their show's seat changes, and the seat service pushes small deltas as holds happen. Stale is fine; the hold decides.
3. Spread the collisions. Offer "best available" as the default: the server picks adjacent seats from a pool, rotating where it starts, so thousands of people are not all fighting for row J, seats 10 to 14. Users who insist on picking still can.
4. Protect the hot show. Rate-limit holds per user and per device (R17), cap seats per booking, and shorten the hold to five minutes during the opening. For the very hottest shows, route all holds for that show to a single writer — one thread that applies holds in order from a queue, so they never wait on each other's locks. It is R18's one-owner idea, applied to one show.

What's being probed: protecting a hotspot without weakening the correctness guarantee.

Concept — admission control and fairness. When demand is a hundred times supply, most people will be disappointed whatever you do. The design choice is how: quickly and fairly, or slowly after a timeout. A waiting room bounds the load to what the system can serve, and random admission at the start is fairer than a race won by whoever has the fastest connection or a script. R14 covers admission control in general; Amazon's flash sale, next, reuses the same layers.

Takeaway: Admit people at the rate you can serve them, spread their choices, and keep the database as the only judge.

Deep dive — cinemas with their own systems

Interviewer: Some chains also sell at their box office and on their own app.

You: Then their system owns those seats, and our database is only a copy. A hold becomes a call to the chain's API — try — with their own expiry, and confirm and cancel are calls too. It is the same TCC shape, but across a company boundary, so:

• every call carries an idempotency key, because their API may time out after doing the work;
• a timeout is unknown: query the reservation before retrying or giving up;
• our hold expiry must be shorter than theirs, so we never sell a seat they have already released;
• their seat map is synced into our cache from their change feed or by polling;
• a nightly reconciliation compares our bookings with theirs and flags every difference.

When another system owns the inventory, our database is a cache of it, and their reserve call is the real hold.

Failures and scale

Interviewer: What breaks, and how?

You:

Curveballs

Interviewer: The cinema wants no single empty seat left between booked seats.

You: The seat map is checked on the client for quick feedback, and the rule is checked again in the hold transaction, which can see neighbouring seats on the same partition and reject a hold that strands one seat. Both, because the client can be bypassed.

Interviewer: Someone holds seats over and over, never paying, to block them.

You: Limit active holds per user and device, slow down repeat holders, and shorten holds for accounts that never pay. During openings, holds are only possible with a waiting-room token.

Interviewer: The cinema cancels a show.

You: A bulk saga: mark the show cancelled, move every booking to REFUND_PENDING through the outbox, and refund each one with an idempotency key per booking, so a crash halfway through resumes without refunding anyone twice. Reconciliation confirms every refund settled.

Interviewer: A user cancels a booking — does the seat go back on sale?

You: Yes, within the cinema's rules: the booking becomes CANCELLED, the seat row goes back to AVAILABLE in the same transaction, and the refund runs from the outbox event. Close to showtime the seat may be sold again in minutes, so its history stays on the booking rows, not the seat row.

Quick reference

The 60-second version

Selling each seat once is one atomic statement: a conditional update that moves all the requested seats from available — or expired-held — to held, on the show's own partition, with the row count telling you whether you got them all; any shortfall rolls back, so groups are never split. Holds last ten minutes and expiry is part of the claim, so no background job is needed for correctness; a sweeper just tidies the seat map. Payment is a saga with the hold id as the idempotency key: confirm is conditional on the hold still owning the seats, duplicate callbacks change nothing, timeouts are resolved by asking the PSP, and a payment that lands after the hold lapsed is refunded automatically. Seat maps are cached and pushed, and allowed to be slightly stale, because the hold decides. Booking-open stampedes meet a waiting room with random admission, best-available allocation, rate limits, shorter holds and a single writer for the hottest shows. Where a chain owns its seats, the hold becomes a try–confirm–cancel call to its system, reconciled every night.

Walkthrough 2 — Amazon, a marketplace with checkout and inventory

Building on BookMyShow: every seat was a distinct row. Here units are interchangeable, so the claim becomes a conditional decrement of a stock count, and the hold sits inside a longer checkout saga across inventory, orders and payment. Most of the system — catalog, search, cart — is deliberately not strict, and the interesting question is where loose data has to become strict.

Scope and requirements

Interviewer: Let's design Amazon.

You: It's huge, so two minutes of narrowing. First-party store, marketplace, or both — because a marketplace adds per-seller inventory and split shipments? Which part matters most — browse and search, cart, or checkout and inventory? Volumes, and any big sale events? Single country or global? Can we briefly oversell and apologise, or never? Payments ourselves or through a PSP? Fulfilment and returns in scope?

Interviewer: Global marketplace at roughly Amazon scale. Focus on checkout and inventory, but cover the whole thing.

You: Then: first- and third-party sellers on one catalog; 300M MAU, 100M DAU; 500M SKUs; 10M orders a normal day, 5x on a sale day with 20x bursts on a few hot SKUs; several regions with a home region per user; payments through an external PSP with authorise-then-capture, storing only tokens. Fulfilment, shipping, returns, seller tools and ads consume our events. Never oversell a confirmed order; a cart that turns out to be stale is acceptable. That sentence splits the system into a part that must be correct and a part that can be approximate.

Functional: browse product pages, search with filters and facets, a cart that persists across devices and merges guest carts on login, checkout with address and payment, reserve stock during checkout, authorise payment and capture at shipment, order history and status notifications, recommendations. Non-functional, per component: product page and search p99 under 200 ms at 99.99%; the cart always writable, even during partial outages; checkout correct over fast, p99 under 1 s excluding the PSP; exactly-once business effect for orders and payments under retries; 5x without manual intervention; never lose a confirmed order; search freshness within seconds to a minute.

Interviewer: Always-writable cart but strongly consistent inventory. Isn't that contradictory? Pick one.

You: They are different data with different costs of being wrong. A lost cart write means a shopper re-adds an item. Wrong inventory means selling units we don't have, cancelling orders and breaking trust. So AP for the cart, CP for inventory and orders. The cart is only a wish list until checkout re-checks price and stock against the strongly consistent stores.

What's being probed: whether you apply CAP per data type, not per system.

Concept — Consistency by cost of a wrong answer. R0 covers CAP. The senior move is to choose it per kind of data: a cart is a sticky note — if two copies disagree, merge them; a stock count is a bank balance — two people cannot both take the last unit. Then put an explicit validation step (checkout) where loose data becomes strict data. "Exactly-once business effect" is the precise phrase: delivery is at-least-once, but the outcome — one order, one charge — happens once.

Takeaway: Choose consistency per data type by asking what a wrong answer costs.

Capacity

Interviewer: Size it.

You: 20 product views, 10 searches and 3 cart writes per DAU; 3 items per order; 10 KB product records, 8 images at ~200 KB, 5 KB orders, 2 KB carts.

Order QPS is small in absolute terms. The hard part is not throughput, it is contention on a few hot inventory rows.

Storage: catalog 5 TB (15 TB replicated); images 800 TB in S3 behind a CDN; orders 50 GB a day, ~90 TB over five years; carts ~400 GB; search index ~1 TB primary; Kafka ~500 GB a day for 7 days. Bandwidth: product JSON ~2.3 GB/s peak at origin, mostly absorbed by cache; images ~57 GB/s peak, which must come from the CDN. Cache: if 20M of 500M SKUs take 90% of views, the hot set is 200 GB — about ten Redis nodes plus replicas. Availability hints for every SKU fit in 25 GB.

What's being probed: whether you can draw a design conclusion from the math.

Concept — Average, peak, hotspot. Divide daily totals by ~86,400, multiply by a peak factor, then ask whether the load spreads. 5K orders a second over millions of SKUs is easy. 5K a second on one SKU is thousands of writers fighting for one row, and horizontal scaling doesn't fix it because a single row lives on a single machine. R11's hot-row demo shows the effect.

Takeaway: Total load sizes the fleet; hotspot load shapes the design.

API

Interviewer: Sketch the external API.

You: Through the gateway, OAuth bearer tokens, cursor pagination, and an Idempotency-Key on anything that changes money or stock.

POST /v1/orders
Idempotency-Key: 6f1c2a9e-8d1b-4a7e-9c55-2b0f3d1e7a10
{ "checkoutSessionId": "cs_01HZX3...", "shippingAddressId": "addr_88",
  "paymentMethodToken": "pm_tok_4421", "expectedTotal": { "amount": 459900, "currency": "INR" } }

HTTP/1.1 202 Accepted
{ "orderId": "ord_01HZX4K...", "status": "PENDING_PAYMENT",
  "statusUrl": "/v1/orders/ord_01HZX4K...", "reservationExpiresAt": "2026-10-02T10:15:00Z" }

Same key and same body returns the same response; same key with a different body is 422 — R3's request-hash check. expectedTotal lets the server refuse with 409 PRICE_CHANGED rather than charge a price the user didn't see. Money is an integer in minor units. 202 because payment may take seconds and I won't hold a connection open on a third party; the client polls statusUrl or gets a push. Cart writes carry a cartVersion, but item-level changes commute, so a stale version is merged, not rejected — rejecting would make the cart feel broken on flaky networks. Internally, gRPC with mTLS, and the order's key travels downstream. Put the correctness guarantees in the API contract, not just in the backend — each of these is something you can point to when the interviewer asks about a retry or a price change.

Data model

Interviewer: Data model, and where each entity lives.

You:

Every query is a key lookup or a per-user range. The address is a snapshot on the order, because the user may edit their address book later and the order must remember where it shipped.

Interviewer: Six stores. Why not one big Postgres?

You: For a small shop I'd start there with JSONB. At this scale the patterns genuinely conflict: a huge flexible-schema read-heavy catalog, carts that need availability across regions, inventory and orders that need transactions, and search that needs an inverted index over 500M documents with facets. Each fact has exactly one source of truth; everything else — search above all — is a derived view I can rebuild from change events.

What's being probed: whether you co-locate data that must change together.

Concept — Co-location turns distributed into local. The reservation lives on the same shard as its inventory row, and the idempotency record and outbox live with the order. That turns "update stock and record the hold" and "create the order and its event" into single local ACID transactions instead of R2's distributed ones. Polyglot persistence is safe only under the rule one source of truth per fact.

Takeaway: Key by your queries; put things that must change together on the same shard.

High-level design

Interviewer: Draw it.

You: Browse runs left to right through CDN, gateway, catalog and cache. Checkout runs through the order service, which orchestrates inventory and payment. Everything that changes publishes events. In the diagram, run the browse flow and then the checkout flow, and compare which arrows are synchronous calls and which are events.

The CDN serves images and caches anonymous product JSON for ~60 s. The gateway handles auth, per-user and per-IP rate limits, bot protection, and queues or sheds load during flash sales. Catalog owns products and offers behind a Redis read-through cache. Search queries Elasticsearch, re-ranks and hydrates from the catalog cache; an indexer consumes change events. Cart is stateless over DynamoDB. The order service prices, orchestrates the saga, and owns order state and idempotency. Inventory exposes reserve, commit, release and runs the expiry sweeper. Payment wraps the PSP — authorise, capture, void, refund — passing idempotency keys through. Kafka carries catalog-changes, inventory-changes, order-events, payment-events, fed by outboxes.

One owner per piece of data: the order service asks inventory to reserve; it never updates stock rows itself. Synchronous calls where the caller needs an answer now; events where others just need to know.

Interviewer: A shopper searches, opens a product and adds it to the cart.

You: Search builds a query — full-text on title and brand, filters as non-scoring clauses, aggregations for facets — re-ranks by relevance, sales velocity, stock and personalisation, and hydrates from cache. The product page comes from the CDN on a hit; on a miss the catalog reads Redis, then the DB, filling Redis with a TTL. The availability shown — "Only 3 left" — is a cached hint, not a guarantee. Add-to-cart fetches the current price and hint, does not reserve stock, and writes SET items.#sku = :qty to DynamoDB in the user's region.

Deep dive — catalog and search

Interviewer: 500M products, thousands of categories. Store attributes and serve pages fast.

You: A laptop has RAM and CPU; a shirt has size and colour. One wide relational table means hundreds of sparse columns; entity-attribute-value tables mean many joins per page. A document store keyed by productId, with a versioned attribute schema per category validated on write, gives one read per product. Variants live inside the parent; offers live separately because they change far more often.

Serving is R12's layering: images cached forever under content-hashed URLs; anonymous JSON ~60 s at the CDN; Redis with ~10 min TTL for products and ~30 s for offers. Catalog writes flow through CDC; a consumer deletes the Redis key and purges the CDN path, with TTL as the safety net. Single-flight per key and early probabilistic refresh stop stampedes. Personalised bits — Prime price, delivery date to your pin code — are fetched separately and never cached at the CDN.

Interviewer: How does search stay in sync?

You: Catalog, offer and inventory DBs emit changes via CDC or outbox into Kafka (R4); an indexer builds a denormalised document and upserts it with external versioning, so an older event arriving late cannot overwrite a newer document. A full rebuild indexes into a new index from a snapshot, replays Kafka from the snapshot offset, and swaps an alias. Shards spread by document id with scatter-gather queries, split by category or locale at extreme size. Ranking is BM25 retrieval of ~1,000 per shard, a learned re-rank, then business rules. And I index only inStock or a bucketed count, updated when it crosses a threshold, not every unit change. Feed derived indexes from the database's change log, versioned — never by dual-writing.

Deep dive — the cart

Interviewer: You said the cart must never fail. How?

You: A lost add-to-cart is lost revenue. Amazon's 2007 Dynamo paper publicly used the cart as its motivating case: accept writes during partitions and reconcile later. DynamoDB or Cassandra keyed by cartId, replicated across regions, storing a map skuId → {qty, updatedAt} with item-level updates so concurrent adds to different SKUs never clash.

• Different SKUs: union.
• Same SKU, concurrent quantity changes: last-writer-wins on that item.
• Deletes: a tombstone {qty: 0, updatedAt}, so a stale replica cannot resurrect the item — the anomaly the Dynamo paper describes.
• Guest-to-user merge on login: union, max quantity for the same SKU.

What's being probed: conflict resolution in an AP store, especially resurrected deletes.

Concept — Per-item merges and tombstones. If two replicas accept writes independently you need a rule to combine them. A map of items makes most conflicts disappear because edits to different items simply combine. Deletes are the trap: remove an item outright and a replica that never saw the delete still has it, so a union brings it back. A tombstone records "deleted at T" and beats older adds — writing "cancelled" on the list instead of erasing the line. R6 covers anti-entropy between replicas.

Takeaway: Make conflicts per item, and record deletes as tombstones, never as absence.

Deep dive — checkout as a saga

Interviewer: Checkout touches orders, inventory and payment. Keep it consistent.

You: Three services, three databases, and an external PSP. 2PC is out — the PSP can't join, and R2 explains why you avoid it anyway. Choreography loses the shape of the flow. Orchestration in the order service — or Temporal or Step Functions — gives one state machine with persisted state after each step. This is R5's saga; R27 describes the payment platform on the other side of step 3.

PENDING -> INVENTORY_RESERVED -> PAYMENT_AUTHORIZED -> CONFIRMED -> SHIPPED -> DELIVERED
   |               |                    |
CANCELLED     OUT_OF_STOCK       PAYMENT_FAILED (release stock)

The happy path: insert the idempotency record and the PENDING order in one transaction, or return the stored response if the key exists. Snapshot the cart and re-price every line; if the total differs from expectedTotal, 409. Reserve each line with a 15-minute TTL. Authorise with the same idempotency key. On success, set CONFIRMED, commit the reservation, and write OrderPlaced to the outbox in the same transaction; CDC relays it, and cart cleanup, notifications and fulfilment consume it.

Rules that make it safe: every step and compensation idempotent, keyed by orderId plus step; state and outgoing events written together; a timeout per step; the reservation TTL is the backstop if everything else fails — stock always comes back; and authorising at order time with capture at shipment means a cancellation before shipping is a cheap void, not a refund.

What's being probed: whether you know why 2PC fails here, how sagas compensate, and where the pivot is.

Concept — Saga, pivot, outbox. R5 teaches the saga in full: local steps, each with a business-level undo, run in reverse on failure. The pivot is the step after which you only move forward — here, confirmation. R4's outbox solves the separate problem that you cannot atomically write your database and publish to Kafka. Say both names and the pivot, and point at the TTL as the last line of defence.

Concept — this saga is TCC. Look at steps 2, 3 and 5: reserve stock with a TTL, authorise the card, capture at shipment. Each participant offers a hold, the order service decides, and every hold expires on its own if the decision never arrives. That is try–confirm–cancel: try = reserve and authorise, confirm = commit the stock and capture the card, cancel = release and void. Naming it scores, because it explains why the undo here always works — a void releases a hold that was placed to be released — and it raises the two rules interviewers probe next: a cancel that arrives before its try (empty rollback) and a try that arrives after its cancel (suspension). R2 works through the checkout as TCC in full.

Takeaway: Sagas replace rollback with compensation; the outbox makes "save and announce" atomic. And where every step has a natural hold, the saga is TCC.

Deep dive — no overselling

Interviewer: Two buyers race for the last unit.

You: First, when do we touch stock? Decrementing at add-to-cart lets abandoned carts hoard inventory. Decrementing at payment success means people pay and then get cancelled. Reserve at checkout with a 10–15 minute TTL and commit on payment. And the reserve is one atomic statement — no read-then-write in application code:

-- Reserve (atomic, no oversell)
UPDATE inventory
   SET reserved = reserved + :qty, version = version + 1
 WHERE sku_id = :sku AND warehouse_id = :wh
   AND on_hand - reserved >= :qty;
-- rows_affected = 0  => out of stock
INSERT INTO reservation (id, sku_id, wh, qty, order_id, status, expires_at)
VALUES (:rid, :sku, :wh, :qty, :order, 'HELD', now() + interval '15 minutes');
-- both in one local transaction on the SKU's shard

Commit sets on_hand -= q, reserved -= q and marks the reservation COMMITTED. Release sets reserved -= q, guarded by status so it runs at most once. The sweeper runs SELECT … WHERE status='HELD' AND expires_at < now() LIMIT 500 FOR UPDATE SKIP LOCKED and releases. Multi-line orders across shards reserve each line independently and release the winners if any line fails — a mini-saga. Optimistic locking on version also works, but at a hot row it burns retries; the guarded update needs no retry loop.

What's being probed: race conditions, atomic check-and-set, and the reservation lifecycle.

Concept — Put the check inside the write. "Read stock = 1, check > 0, write 0" lets two requests both read 1. The WHERE on_hand - reserved >= qty clause is evaluated while the row is locked, so only one request passes for the last unit, and rows-affected tells you who won. A TTL reservation is a theatre seat held for 15 minutes; SKIP LOCKED lets several sweepers share work without blocking. R8 makes the same point: a conditional update decides a race with no lock service at all.

Takeaway: Put the check inside the write; let the database decide the race.

Interviewer: Prime Day. 1,000 discounted TVs, 1M users clicking "buy" in the same second, all on one row.

You: Correctness still holds, but every request queues for one row lock, latency explodes, and the shard can fall over and take other SKUs with it. So I add layers in front, without changing who has the final say:

• A virtual waiting room at the edge admits roughly stock times a small factor.
• A Redis pre-check: DECR stock:sku, reject fast below zero and INCR back. A gate, not the truth.
• Split the hot row into N buckets, each with part of the units; pick one at random, try another on failure, rebalance as they drain.
• A single writer per hot SKU consuming a queue partition and applying reservations in batches — serialised instead of fighting over locks.
• A shorter TTL, say 5 minutes, so abandoned holds return quickly.

Most of the million are told "sold out" in milliseconds; only a few thousand reach the database.

What's being probed: whether you can protect a hotspot without giving up correctness.

Concept — Shed load before the source of truth. When demand vastly exceeds supply, most requests will fail anyway, so make them fail early and cheaply. Waiting rooms and Redis gates turn away the many; bucketed rows turn one lock into N; a single-writer queue replaces contention with order. The database guard stays the final judge, so a Redis glitch can reject a buyer but can never oversell a unit. R14 covers admission control.

Takeaway: Fast gates reject the many; the database decides the few.

Deep dive — idempotency end to end

Interviewer: Flaky network, the response is lost and the app retries — or the user double-taps.

You: The client mints one key per "Place order" intent and reuses it. R3 has the full pattern; the shape here is an insert of (user, key, hash(body), IN_PROGRESS) under a unique constraint, in the same transaction as the PENDING order. On conflict, a different hash returns 422, a DONE row returns the stored response byte for byte, and anything else returns 409 in progress so the client retries later. The row flips to DONE with the response once the saga settles.

The key must flow all the way down: inventory reserves keyed by orderId, payment sends orderId:attempt as the PSP idempotency key — major PSPs publicly support them — and Kafka consumers dedupe by event id because delivery is at-least-once (R9).

Failures and scale

Interviewer: What breaks, and what fails badly?

You:

Interviewer: A region goes down. Carts, inventory, orders?

You: Stateless services are active-active behind latency-based routing. Catalog has one writer region and async replicas everywhere; search runs a cluster per region fed from mirrored Kafka. Carts are active-active global tables with item-level LWW and tombstones. Inventory must never be written in two regions for the same unit, so each warehouse's stock has a home region and reservations route there — which also matches fulfilment. Orders are written in the user's home region; on failover we promote a replica (RPO is the replication lag, seconds) and sweepers plus PSP reconciliation resolve mid-saga orders. RTO minutes for browse and cart, ~15 minutes for checkout. Degrade recommendations, then personalisation, then facets, before touching cart or checkout.

Curveballs

Interviewer: Payment times out. You don't know if the customer was charged.

A timeout is "unknown", not "failed"

You: We never guess. The payment service queries the PSP with the same idempotency key and acts on the real answer, while the reservation holds the stock. On a decline, the orchestrator compensates in reverse: release the reservation, mark PAYMENT_FAILED, emit the event through the outbox. The client may retry with another card inside the reservation window, with a new key for the new attempt. If the outcome is still unknown, reconciliation against the PSP's settlement reports voids or confirms — R6 carries that loop.

Interviewer: Rapid fire. Why not reserve at add-to-cart? Price changes during checkout? Orchestration over choreography? Catalog DB down?

You: Most carts are abandoned, so reserving on add starves real buyers — except scarce drops like tickets, where a short hold makes sense. Price changes return 409 PRICE_CHANGED from expectedTotal. Orchestration because checkout has several steps, timeouts and compensations that need one inspectable state machine; choreography is fine for two steps. Catalog DB down: pages keep rendering from CDN and Redis, stale-on-error, and only edits block. Every answer points back to a mechanism already on the board — persisted saga state, then the TTL, then reconciliation — which shows the design is coherent rather than patched on the spot.

Quick reference

The 60-second version

Browse is a caching problem: the catalog in a document store behind Redis and a CDN, images in S3, search in Elasticsearch fed by CDC through Kafka with versioned upserts and two-stage ranking. The cart is an always-writable, multi-region key-value map that merges per item with tombstones, in the spirit of Dynamo. Checkout is an orchestrated saga — create the order, reserve stock with a TTL, authorise payment, confirm — with an idempotent compensation for every step and the confirm as the pivot. Overselling is prevented by one atomic conditional update on the SKU's row; flash sales add a waiting room, a Redis gate, bucketed rows and a single writer. Idempotency keys flow from the API to inventory and the PSP, and the outbox makes every state change and its event atomic. Carts and stateless services run active-active; inventory and orders are homed to one region because they cannot merge.

Walkthrough 3 — Uber, real-time location and matching

Building on BookMyShow and Amazon: now the scarce thing moves. A driver must be found before they can be claimed, the hold — an offer — lasts seconds rather than minutes, and finding the right driver means a geospatial search over hundreds of thousands of location updates a second. The claim itself is the same move as before: a compare-and-set that only one request can win.

Scope and requirements

Interviewer: Design Uber. Riders request rides, drivers pick them up.

You: Scope first. Standard rides only, or pool, delivery, freight — because shared rides turn matching into a routing optimisation? Global or one country? How often do driver apps send location — that sets the write rate of the hottest path. Nearest-driver or a global optimisation? Maps and ETA ourselves or as a service? Surge in scope? Payments through a PSP? Which latency matters most?

Interviewer: Standard rides, global, you decide the rest.

You: Standard rides; pool, delivery and freight out. Global, sharded by city. Drivers send location every 4 seconds while online. Near-greedy matching: nearby available drivers, ranked by road ETA, offered one at a time. Maps, routing and ETA are an internal service. Surge per geospatial cell, refreshed every 1–2 minutes. Payment after the trip through an external PSP using card tokens.

Interviewer: Why not a proper global optimisation from the start? It gives better matches.

You: At high density it does, but it adds a 1–2 s batching delay and a solver per city cell. For a p95 match under ~10 s, greedy is the simplest thing that works — and I'll make "choose a driver" a pluggable step, so batch matching is an extension for dense cities at peak, not a rewrite.

What's being probed: whether you can defend a simpler choice without dismissing the better one.

Concept — Greedy versus batch matching. Greedy gives each request the best driver right now — fast and easy to reason about, but it may take the one driver who was perfect for a request arriving half a second later. Batch matching waits a short window and solves an assignment problem minimising total ETA — seating a whole group well rather than guests one by one. The trade is a little latency for better global efficiency.

Takeaway: Start greedy, keep the matcher pluggable, and name batch matching as the upgrade.

Interviewer: Requirements.

You: Functional: drivers go online and stream location; riders see nearby cars, a fare estimate with surge and a pickup ETA; request, match, accept within a timeout; live updates through arrival, start and end; a tracked trip lifecycle; fare computed at the end and charged to the saved method, with driver earnings recorded; history and receipts. Non-functional: nearby search under 100 ms; p95 match under ~10 s including acceptance; driver position on the rider's map no older than ~5 s; 99.99% for request and match, with each city surviving another region's failure; a driver never on two trips, a rider never charged twice; sharp regional peaks. Out of scope: map building, pool, onboarding, deep fraud and ratings, support tooling.

Interviewer: You listed "durable" and "lossy" together. Which data can we lose?

You: Location pings. A ping is worthless four seconds later because the next one replaces it. Trips and payments can't be lost and must be strongly consistent — a trip moving to MATCHED twice or a charge going out twice is real-world harm. So two classes of storage on purpose: fast, volatile, latest-only for location; durable and transactional for trips and money. Analytical history sits in a third tier: kept, but allowed to arrive late.

Capacity

Interviewer: What's the hottest path?

You: Driver locations, by far. 20M trips a day at 5x peak; 1.5M drivers online at peak, 750K average; 100-byte binary pings; 20-minute trips; ~10 fare estimates per request; 2 KB trip records; ~50K connections per gateway node.

375K location writes against 1.2K ride requests — about 300 to 1 — so the location path needs its own design. And 450 MB of live state fits on one machine; we shard for write throughput and blast radius, not size.

Storage: trips 40 GB a day, ~75 TB over five years. Route points during trips ~300 GB a day, ~550 TB over five years, tiered to object storage after 90 days. Raw pings ~1.6 TB a day go to the data lake in compressed columnar form, never to an operational store.

What's being probed: whether you notice the dominant load is message count, not bytes.

Concept — QPS versus bandwidth. 300 Mbps fits on one network card, so the bytes look easy. But 375K separate messages a second each need parsing, validation, an index update and maybe a forward, and per-message overhead — TLS, headers, a database transaction — multiplies by the count. When messages are tiny and frequent, optimise cost per message: persistent connections, binary frames, stores built for small overwrites.

Takeaway: Tiny frequent messages are a QPS problem, not a bandwidth problem.

API

Interviewer: Is it all REST?

You: Two transports, as R1 lays out. Request/response actions over HTTPS; location and server pushes over a persistent WebSocket (or gRPC stream or MQTT). Short-lived OAuth tokens with different scopes for riders and drivers.

A ping is protobuf on the wire — driverId, lat/lng, heading, speed, accuracy, a timestamp and a sequence number ({"type": "loc", "driverId": "d_771", "lat": 12.97161, "lng": 77.59462, "seq": 48121, …}) — and the server drops stale or out-of-order ones.

POST /v1/trips
Idempotency-Key: 6f1c2a9e-7b0d-4c1e-9a55-2f3e8d1b4c77
{ "quoteId": "q_9Kd2...signed", "pickup": {"lat": 12.9716, "lng": 77.5946},
  "dropoff": {"lat": 12.9352, "lng": 77.6245}, "productType": "UberX", "paymentMethodId": "pm_tok_123" }

201 Created
{ "tripId": "t_01HF8...", "state": "REQUESTED",
  "fare": {"amount_minor": 31200, "currency": "INR", "surge": 1.3}, "cityId": "blr" }

The idempotency key stops a double booking from a double tap (R3); accept takes one too, and the PSP charge uses tripId. The signed quote binds the price: the rider pays what they saw, and the server doesn't need to store every estimate. History uses opaque cursors.

Data model

Interviewer: Main entities and stores?

You:

Interviewer: PostGIS does spatial queries well. Why add Redis? One fewer system.

You: At a few thousand drivers in one city, PostGIS is my choice. At 375K updates a second to rows overwritten every 4 s, each update rewrites WAL, table pages, index pages and replicas, for data we don't need durably. In memory with a TTL, writes are sub-millisecond and dead apps vanish on their own. If a shard dies we lose one cycle — drivers resend within 4 s. History still exists, in Kafka and the lake.

What's being probed: whether you can explain write amplification and name the scale at which the alternative wins.

Concept — Don't pay durability for data that heals itself. One logical write in a database causes several physical ones: the log, the page, each index, each replica. That is the right price for a trip. For a value overwritten in 4 seconds you pay it 375K times a second for nothing. Ask: "If I lose this value, how long until it's correct again?" For a GPS ping the answer is 4 seconds.

Takeaway: Don't pay durability costs for data that heals itself in seconds.

High-level design

Interviewer: Draw the architecture.

You: Two entry points — an API gateway for REST and a WebSocket gateway for streams. A location service owns the hot path, a trip service owns the state machine, dispatch owns matching, and Kafka joins everything that can be asynchronous. In the diagram, play the four flows — ping, match, live tracking, payment — and notice that only the first three are on anyone's waiting path.

The WebSocket gateway terminates ~2M connections, with a Redis session registry userId → node so any service can push to a user. The location service validates pings (sequence, speed sanity, accuracy), writes the geo index, publishes to Kafka, and forwards the position to the rider during a trip. The trip service is the only writer of trip state; every transition is a conditional update that emits an event. Dispatch finds candidates, ranks by ETA, offers with timeouts and locks the driver. ETA/routing is a black box with batch one-to-many calls. Pricing combines a base model with the cell's surge and signs quotes. A surge worker streams over Kafka. The payment service consumes TripCompleted. APNs/FCM cover backgrounded apps; the data lake feeds ML, disputes and audits.

The two flows I'd narrate:

• Location ping: gateway forwards to the city's location instance; drop if seq is stale or the jump is physically impossible; compute the H3 cell (resolution ~9); if the cell changed, move the driver id between cell sets; always overwrite driver:{id} and refresh a 30 s TTL; publish to driver-locations keyed by driverId; if on a trip, push to the rider.
• Request and match: estimate reads surge and ETA and returns a signed quote; POST /v1/trips validates the quote and inserts REQUESTED; dispatch reads the pickup cell and its rings, keeps AVAILABLE drivers of the right product, batch-calls ETA for the top ~20, CASes the best AVAILABLE → OFFERED(offerId) for 15 s and offers; decline or timeout releases and tries the next, widening after N failures or ~60 s; accept CASes OFFERED → ON_TRIP and the trip row goes REQUESTED → MATCHED; the rider hears over the WebSocket.

During the trip, the rider app interpolates between 4-second points and reconnects with GET /v1/trips/{id}; route points append to Cassandra for the fare and receipt.

What's being probed: whether ownership is clear and the synchronous path is short.

Concept — Latest value plus log. Hot-path readers want only "where is driver 771 now?"; analysts and models want every value over time. One store serving both is bad at both. So overwrite the latest value in a fast key-value store and append every event to a durable log — the log is the history, the key-value store is a rebuildable cache of "now". Everything off the rider's waiting path — surge, payment, analytics, receipts — goes through events (R9).

Takeaway: Overwrite "now" in memory; append history to a log.

Deep dive — location ingestion and the geo index

Interviewer: 375K writes a second. Options?

You: HTTP POST per ping pays TLS and headers each time and can't push offers back. SQL per ping is write amplification. So a persistent connection, Redis for latest state, Kafka for history — plus cuts to per-message work: keep only the newest ping per driver per interval; adapt frequency (slower when parked, faster on a trip); touch the cell set only when the cell changes, so most pings are one hash overwrite; TTL plus a periodic sweep removes dead apps from cell sets.

Interviewer: "Available drivers within 2 km", thousands of times a second, while the index changes 375K times a second?

You:

H3, which Uber has publicly described building and open-sourcing for indexing supply, demand and pricing. Resolution ~9 (edge ~170 m) for search; ~7–8 for surge so each cell has enough events to be stable.

nearby(lat, lng, product):
  c = h3.latLngToCell(lat, lng, 9)
  for k in 1..4:
     ids = union(SMEMBERS cell:{x} for x in h3.gridDisk(c, k))
     cands = [d for d in ids if status(d)==AVAILABLE and d.product==product]
     if len(cands) >= 20: break
  return top 20 by haversine(lat,lng,d) -> send to ETA for ranking

A fixed grid never rebalances; a driver moves between two sets only when crossing an edge.

Interviewer: The rider stands on a cell edge and the nearest driver is 50 m away in the next cell.

You: I never search one cell. k = 1 is the pickup cell plus its six neighbours, which approximates a circle wherever in the cell the rider stands, and the final order is by road ETA anyway.

What's being probed: whether you know how a spatial index turns proximity into key lookups, and its edge effect.

Concept — Grid indexing with a coarse and a fine filter. Databases are good at "find by key", not "find near a point". A grid gives every cell an id, so "near me" becomes "my cell and its neighbours". Any grid creates artificial edges, so cells are only the coarse filter that shrinks the candidate set; exact distance or road ETA is the fine filter. Hexagons help because all six neighbours are the same distance away — square tiles have farther corner neighbours.

Takeaway: Turn "near" into "same or neighbouring cell", then rank by true distance.

Deep dive — matching without double assignment

Interviewer: Two riders request at once and both dispatchers pick the same driver.

The bug: check then act

You: Two layers. First, a driver lock in Redis: a Lua compare-and-set or SET driver:{id}:state OFFERED:{offerId} NX PX 15000. Only one dispatcher moves a driver out of AVAILABLE; the other moves to its next candidate; the expiry releases the driver if a dispatcher crashes. Second, the trip row is the source of truth:

UPDATE trips SET state='MATCHED', driver_id=:d, version=version+1
WHERE trip_id=:t AND state='REQUESTED' AND version=:v;
-- 0 rows -> someone else matched or rider cancelled; release driver

plus one active trip per driver — an active_trip row keyed by driverId in the same transaction. The Redis lock is fast but volatile; the database row is durable and has the final word. Accept is idempotent: a retried accept(offerId) sees ON_TRIP for the same offer and returns success.

What's being probed: whether you prevent a race with atomic operations.

Concept — Compare-and-set in two layers. CAS merges check and write into one atomic step — "set OFFERED only if currently AVAILABLE" — so only one caller succeeds. The SQL form is optimistic locking on state and version; zero rows means you lost. The lock expiry stops a crashed holder blocking a driver forever. R8 covers both, and its advice applies directly: Redis is fine for stopping duplicate work, the durable store decides.

Takeaway: Never check-then-act; make the check and the write one step, and let the durable store decide.

Interviewer: One offer at a time is slow. Three declines and the rider waits 45 s. Why not offer to five and take the first accept?

You: It cuts time-to-match, but four drivers "accept" and lose, and drivers who keep losing stop accepting; it also holds five drivers in OFFERED for one request. I'd keep one at a time, shorten the timeout in dense areas, and rank likely-decliners later. At density the bigger win is batch matching every 1–2 s with an assignment solver (Hungarian algorithm or min-cost flow), which Uber has publicly described moving towards — a contained change because the matcher is pluggable. Speed for riders must not burn out drivers; I'd measure acceptance rate alongside time-to-match.

Deep dive — trip lifecycle, surge, payment

Interviewer: How do you keep trip state correct across services?

You: The trip service owns a strict state machine.

REQUESTED -> MATCHED -> DRIVER_ARRIVED -> IN_PROGRESS -> COMPLETED
    |            |             |                |
 CANCELLED   CANCELLED     CANCELLED     (rare) CANCELLED_BY_SUPPORT

Every transition is a conditional update on (state, version), so illegal transitions are rejected; DRIVER_ARRIVED soft-checks the driver is within ~100 m; cancellation fees apply after a grace period. Every transition writes its event through R4's outbox in the same transaction, so payments never miss TripCompleted. Clients receive full state, not deltas, so a missed message heals on the next one.

Interviewer: Open ETA up a little. Then surge.

You: The road network is a graph; plain Dijkstra is too slow at continental scale, so routers precompute — contraction hierarchies or partition-based schemes — and answer in milliseconds. Edge weights come from live and historical speeds off the ping stream, and an ML model corrects the router's ETA for time, weather and pickup type, as Uber has publicly described. Dispatch needs one-to-many matrix calls. Surge is a Flink or Kafka Streams job counting open requests and app opens against available drivers per H3 cell over a sliding 1–5 minute window, computing multiplier = f(demand/supply), smoothed across time and neighbouring cells, capped per city, written to Redis — and locked into the signed quote.

Interviewer: And the payment?

You: The payment service consumes TripCompleted and inserts a payment row with unique (tripId, CHARGE) — if it exists, this is a redelivered event and it stops. It calls the PSP with idempotencyKey = tripId; on success it records CAPTURED, writes double-entry ledger lines for rider debit, driver earning and platform fee, and emits PaymentCaptured. On a timeout it retries with the same key, which is safe; on a decline it marks FAILED and flags the outstanding balance. A daily job reconciles the ledger against PSP settlement reports. At-least-once delivery plus an idempotent receiver at both our database and the PSP gives exactly-once charges — R3, R4 and R6 in sequence, landing on the kind of platform R27 describes.

Deep dive — sharding and hot zones

Interviewer: How do you shard, and what about an airport with 500 waiting drivers?

You: Partition by city. A rider in Bengaluru never needs drivers in Mumbai, and a broken shard hurts one city, not the world. Big cities split further by H3 parent cell; a pickup near a city edge fans out to at most two shards. Hot zones: split hot cells at a finer resolution, pre-scale from the event calendar, rate-limit and sample the nearby-cars map. Airports run a FIFO driver queue in a staging lot — a geo query becomes a queue pop, and the driver who waited longest goes next.

What's being probed: whether your shard key follows query locality and whether you handle skew.

Concept — Locality sharding and a different access pattern for hot spots. R10 covers shard keys; the right one keeps a query's data on one shard, and city does that for ride-hailing while also limiting blast radius. Any key has skew. You fix it by splitting finer, adding replicas, or changing the access pattern entirely — the airport queue replaces a hot geo search with something cheaper and fairer.

Takeaway: Shard by query locality, and treat hot spots with a different access pattern, not just more hardware.

Failures and scale

Interviewer: Where does it break?

You:

Every dependency gets a timeout and a fallback: rebuild what is cheap, recover what is precious — R14 covers the patterns.

Interviewer: A region goes down with 100,000 trips in progress.

You: Each city is homed in one active region with a warm standby (R13). The trip DB replicates across regions; live location doesn't need to, because it rebuilds from pings in seconds. On failure, routing moves the city; drivers reconnect and refill the geo index. For in-flight trips, Uber has publicly described using the driver app as a store of trip state, so the server resumes from the phone's copy plus the replicated DB — covering whatever async replication lost. Charges are owned by the trip's home region, and the PSP key protects any overlap during failover.

Curveballs

Interviewer: The driver's accept confirmation is lost. Then: shared rides? GPS spoofing? What would you alert on?

You: First, a lost accept confirmation is harmless: the retried accept(offerId) sees ON_TRIP for that offer and returns the same success. Pool turns matching into route insertion: add a rider to an existing route only if current riders' detour limits hold, batching requests for a few seconds and scoring insertions with ETA — inside the pluggable matcher. Spoofing: impossible speeds and jumps, map-matching mismatches and sensor inconsistency, flagged to the fraud system rather than blocked in the hot path; the sequence and speed checks already drop the worst. Alerts: time-to-match p50/p95, match rate, offer acceptance, location ingest lag, connections per node, ETA error, payment failure rate, supply/demand per city — all sliced per city, because a global average hides one city that is completely broken (R15).

Quick reference

The 60-second version

Drivers keep a persistent connection and send location every 4 s — ~375K writes a second at peak. A location service overwrites only the latest position in Redis, sharded by city, and streams every ping to Kafka for history, surge and models. Drivers are indexed by H3 cell; a nearby search reads a ring of cells and ranks the top candidates by road ETA. Dispatch offers to one driver at a time; a Redis compare-and-set on driver state plus a conditional update on the trip row prevents double assignment, and accept is idempotent. The trip service owns a strict state machine in sharded SQL and emits every transition through an outbox. Surge is a windowed stream aggregation per cell, locked into a signed quote. Payment runs after TripCompleted, idempotent on tripId in our database and at the PSP, with a ledger and daily reconciliation. Everything is sharded by city, hot zones become queues, and a region failover rebuilds live state from fresh pings and in-flight trips from the phones.

What to say in the interview

"Reservation systems all make one promise — never sell the same thing twice — and I keep it in one place: an atomic conditional write whose row count says who won. Around it go a hold that expires, so abandoned claims come back by themselves; a saga with idempotency keys for the payment in the middle, with a conditional confirm so that a late payment becomes a refund, never a double sale; and, for the stampede, layers that shed load — waiting rooms, Redis gates, split rows, a single writer — but are never allowed to say yes. Then I choose consistency per data type by asking what a wrong answer costs: the seat map, the cart and a driver's dot on the map can be approximate; the seat, the stock count, the trip assignment and the money cannot. And a timeout is treated as unknown until I've asked."