R12 — Caching: when, what, how, and how consistent

R11 took the database as the bottleneck and worked through indexes, replicas, pooling and CQRS. This section takes the next lever on that list, the one that is cheapest to add and easiest to get wrong: keeping a copy of data somewhere closer to the request. R11 already makes the junior version of the argument ("cache reference data, never cache a balance you are about to act on"). Here is the full version: when a cache is worth it, where it goes, how it is filled and written, and exactly how wrong it is allowed to be.

The situation

A payment service validates every outgoing instruction. One of the checks is "is this BIC a real, reachable bank?" The answer lives in a bank directory table, and the table changes perhaps once a day.

At 2,000 payments a second, that is 2,000 identical lookups a second against a table that has not changed since last night. Every one of them pays a network hop and a query. Every one of them competes for connections with the writes that actually matter.

So you keep a copy of the directory in memory. Lookups drop from milliseconds to microseconds, and the database stops seeing the traffic at all.

A week later someone applies the same idea to the account balance check, because it is also read on every payment. Now a payment is authorised against a balance from forty seconds ago, and two payments that each fit the old balance are released against an account that can only cover one of them.

Same technique, same code, opposite outcome. The difference is not in the cache. It is in what the data is used for. That is the whole section in miniature.

The mental model: a cache is a bet

A cache is a bet that a piece of data will be read again before it changes.

You win the bet when reads far outnumber writes, a miss is expensive, and being briefly wrong is cheap. You lose it when the data changes faster than it is read, when the original read was already fast, or when being wrong costs money.

Every caching decision reduces to four questions (should I cache at all?) and three choices (how do I cache it?).

Four questions: should I cache?

All four have to point the same way. A value read a thousand times a second that must be exact at decision time fails question 3, and question 3 is a veto.

Three choices: how do I cache?

1. Where it lives: browser, CDN, gateway, in-process (Caffeine), distributed (Redis), or two levels at once.
2. How it is filled and written: cache-aside, read-through, refresh-ahead, write-through, write-around, write-behind.
3. How it stays fresh: TTL, explicit eviction on write, event- or CDC-driven invalidation, versioned keys.

Most of this section is those three choices in detail. But the rule that governs all of them is short.

The golden rule

Cache reads that inform; never cache reads that decide.

Show the cached price on the product page; re-read the real price when you charge the card. Show the cached balance on the account screen; read the ledger, under a lock, when you authorise a payment against it. The informing read can be a little stale and nobody is harmed. The deciding read is the one the business is relying on, and it goes to the source of truth every time.

Before reading further, run a few data types through the advisor below. Start with the presets: BIC codes, an FX quote, the balance for payment authorisation. Watch which of the four questions flips each verdict, and notice that balance-for-authorisation fails on staleness alone, however hot it is.

Why cache, and the costs you accept

The reasons are concrete and measurable:

• Latency. Redis answers in well under a millisecond. A database query costs tens of milliseconds; a remote API, hundreds.
• Offload. Every hit is a query the database never sees, which leaves its capacity for writes.
• Absorbing spikes. A cache in front of a hot key turns a burst into memory reads.
• Surviving slow dependencies. When a downstream is degraded, serving a stale value can be better than serving an error.
• Cost and quotas. Third-party APIs charge per call and rate-limit you; a cache cuts both.

The costs are just as concrete, and an interviewer wants to hear that you know them: staleness; a second system to run, patch and monitor; invalidation bugs; cold starts and warm-up; memory cost; a new family of failure modes (stampede, avalanche, penetration, hot keys); and harder debugging, because every incident now starts with "is it the cache?"

When not to cache

Say this before you propose a cache, because it shows judgement rather than recall.

• Write-heavy data with little reuse. Audit logs, raw events. Every write invalidates a value nobody read.
• Data that must be exact when you act on it. The account balance for authorisation, inventory at final checkout, payment status before release, permissions right after a revocation. (If you must cache permissions, use a very short TTL plus explicit eviction on revoke.)
• Cheap reads that are already fast. A cache adds a hop and a staleness bug for no gain.
• Highly personalised responses with low reuse. Cache the ingredients that are shared, not the assembled response that is not.
• Sensitive data you do not want copied into another store. If you must, encrypt it and keep the TTL short.

The trap to name: "read on every request" is not the same as "safe to cache". The balance check is the hottest read in a payment system and the one you must never cache. Frequency answers question 1; it says nothing about question 3.

Where to cache: the layers

A request passes through several places that could answer it before it reaches the database. Each one is closer to the user, cheaper per hit, and harder to invalidate than the one behind it.

The same idea goes all the way down to the chip. A CPU looks in its L1 cache, then L2, then L3, then main memory, each larger and slower than the last. Every layer in the table above is that hierarchy repeated at a larger scale: small and fast in front, large and slow behind, and a lookup that falls through until something answers.

In-process versus distributed is the decision that matters most in a service. An in-process cache costs nothing per read but exists once per instance: ten pods means ten copies that can disagree. A distributed cache is one copy everyone shares, at the price of a network hop. That is why very hot data often gets both, a small short-lived L1 in each JVM in front of a shared Redis L2.

Push a request through the layers below. Toggle which layers are warm and watch where it stops, then turn them all cold and watch the full cost land on the database.

What to cache: classify the data first

The useful question is never "should we add a cache?" It is "which class does this data belong to?" Once you know the class, the layer, strategy and TTL mostly follow.

Two rows deserve a second look in a payments interview.

The FX row appears twice, deliberately. An FX rate for display is slow-changing shared data. An FX quote is a third-party response with a validity window set by the provider, and the TTL is that window, not a number you picked. Caching the quote for exactly as long as it is honoured is correct. Caching it longer means pricing a payment at a rate nobody will give you.

The last row is not a gap in the table; it is the most important row. Those values are deciding reads. They are read from the source of truth, under whatever locking the decision needs (R8 covers the locks).

How to fill and write: the strategies

A read strategy decides who loads data into the cache on a miss, and when. A write strategy decides what happens to the cache and the database when the data changes. Real systems pair one of each, so learn them as two short lists and then as pairs.

Read strategies at a glance

Write strategies at a glance

Before the detail, step through the walkthrough below. It plays each strategy as a sequence of arrows between app, cache and database, and its last step crashes the cache during write-behind. Watch for two things: who talks to the database on a miss, and which step a crash would make you lose.

Cache-aside: the default

The application checks the cache. On a hit, it returns. On a miss, the application reads the database, stores the value with a TTL, and returns it.

Why it is the default: it is simple and explicit, only data that is actually requested gets cached, and a cache outage only makes the application slower, never wrong.

What it costs: the first read of every key pays the full miss; every service repeats the loading code; and a value is stale until its TTL expires or someone evicts it.

How to keep it consistent: update the database, delete the key after the commit, and keep a TTL as the safety net. Use it for catalogs, profiles and merchant config. Meta's memcache and Spring's @Cacheable both work this way.

Read-through: the loader moves into the cache

The application only ever talks to the cache. On a miss, the cache's own loader reads the database, stores the value and returns it.

What you gain: the loading logic lives in one place, callers stay simple, and concurrent misses for the same key share a single load.

What you give up: the cache now has to know your data source; the first read still misses; and custom invalidation is harder, because the loader is not your code path.

Use it for reference data, CMS content, config, and DynamoDB fronted by DAX. Caffeine LoadingCache, Hazelcast and Coherence MapLoader, Hibernate's L2 cache and AWS DAX are all read-through.

Refresh-ahead: never let a hot key miss

Each entry has a refresh point, say 80% of its TTL. A read after that point returns the current value immediately and triggers an asynchronous reload in the background.

What you gain: hot keys never miss, latency stays flat, and there is no stampede at expiry because the key never actually expires while it is being read.

What you give up: reloads are wasted on keys nobody reads again, and a value can still be up to one refresh interval stale.

This is the right strategy for the BIC directory and the holiday calendars: hot, predictable, and read on every payment. It also suits reference FX rates, feature flags and a home page. Caffeine's refreshAfterWrite, Coherence's refresh-ahead factor and the CDN directive stale-while-revalidate are all refresh-ahead.

Write-through: cache and database in step

The application writes to the cache; the cache writes to the database synchronously; only when both succeed does the caller get "ok".

What you gain: cache and database stay in step, and a read straight after a write is a fresh hit.

What you give up: every write pays cache latency plus database latency; rarely-read data fills the cache; and you must decide what happens when the database write fails after the cache write succeeded.

Use it for data that is read immediately after it is written: settings, carts, inventory views. AWS DAX, Hazelcast and Coherence MapStore with a write delay of zero, and Spring's @CachePut, are write-through.

Write-around with evict: the default write path

The application updates the database and commits, then deletes the cache key. The next read misses and reloads the fresh value.

What you gain: fast writes, a cache that only holds data someone actually reads, and a natural partner for cache-aside.

What you give up: the first read after each write misses, and there is one rare race (covered below) whose damage the TTL bounds.

This is the default write path for most services, and the right one for write-once, read-rarely data such as logs, documents and order history. Meta's memcache delete-on-write and Spring's @CacheEvict are write-around.

Write-behind: fast writes, at a price

The application writes to the cache; the entry is marked dirty; the caller gets "ok" at cache speed; a flusher later writes dirty entries to the database in batches.

What you gain: the fastest possible writes, and batching that shields the database: a hundred increments to one counter can become a single database write.

What you give up: if the cache crashes before the flush, those writes are gone.

The database also lags behind, and ordering and retries need care. Use it for counters, likes and views, analytics, activity logs and game scores. Never for money or orders. A ledger entry acknowledged at cache speed and lost in a Redis restart is a payment the customer was told succeeded and the bank has no record of. Redis INCR with a scheduled flush, and Hazelcast MapStore with a write delay above zero, are write-behind.

How they pair up

Side by side

Read the data-loss column first. Only one strategy has anything in it, and that column alone rules write-behind out of every money path.

The quick decision path

Is staleness at decision time unacceptable?  -> don't cache (or write-through + evict, tiny TTL)
Is it write-heavy?                           -> only counters/aggregates via write-behind; else don't cache
Read-heavy and the miss is expensive?        -> cache
   Small, shared, rarely changing?           -> in-process Caffeine + refresh-ahead
   Shared across many instances?             -> Redis (cache-aside); very hot -> two-level (Caffeine L1 + Redis L2)
   Per user?                                 -> Redis, key per user, evict on write
   Public and cacheable over HTTP?           -> CDN / Cache-Control

How the people who run caches at scale do it

These are the designs worth naming in an interview, because each one carries a single lesson.

The payments row is the golden rule written as an architecture. The directory, the calendars and the cut-off times are read on every payment and change on a schedule, so they sit in every JVM and reload overnight. The cut-off times themselves are R26's subject; what matters here is that they are reference data and cache well. The FX quote lives exactly as long as the provider honours it. And the two values that release money, the balance and the limit, are read from the system of record at the moment of decision.

Consistency: two systems, no shared transaction

Here is the part people underestimate. A cache and a database are two systems without a shared transaction. That is the same shape as the dual write in R4: two writes, two systems, and a process that can die or be overtaken between them. There is no ordering of "write the database" and "touch the cache" that is free of a window. Every pattern is a choice about which inconsistency you accept, and for how long.

The write orderings, compared

Why delete rather than update? A delete is idempotent and insensitive to order: however many deletes arrive, in whatever order, the next reader loads the current database value. An update carries a value, and two writers' values can arrive in the wrong order. Deleting turns the cache from a second copy you must keep correct into a copy that rebuilds itself from the truth.

The race that survives the default is worth being able to draw. Reader A misses and reads the old row. Writer B updates the row, commits, and deletes the key. Reader A, slow, now writes the old value into the cache. The cache is stale until the TTL expires. It needs a miss, a write and a slow reader to line up, so it is rare, and the TTL caps how long it lasts. Meta's fix is a lease: on a miss, the cache hands the reader a token, a delete invalidates outstanding tokens, and a set with an invalidated token is refused.

CDC is the outbox idea applied to invalidation. Instead of the application remembering to evict after it commits, a process tails the database's commit log and evicts for every committed change. An eviction cannot be lost to a crash after commit, because the commit log is the durable record of it. R4 covers the outbox and the relay; the same reasoning holds here.

Step through the races below. Each step plays one ordering with two concurrent actors, and the last steps apply the fixes: delete after commit, leases, CDC. Watch where the stale value enters the cache and which fix closes that particular door.

The consistency levels you can offer

Do not promise "consistent". Promise a level, and know how you get it.

Bounded staleness is the level to reach for in payments reference data, because it turns "the cache might be stale" into a number someone can sign off on. "The BIC directory is never more than 24 hours old, and is reloaded on a change event" is a statement an operations team can accept. "It's cached" is not. And the deciding reads sit in the last row: strong, which means not cached at all.

Practical rules

1. Always set a TTL, even with explicit eviction. It bounds every invalidation bug you did not find.
2. Evict after the transaction commits, not inside it. Evict inside it and a reader can refill the cache with pre-commit data, or you evict and the transaction then rolls back.
3. Prefer delete over update on writes.
4. Version your values (row version or updated_at) and refuse to overwrite a newer cached version; or use leases, or SET NX.
5. Local caches on N instances are N copies. Broadcast invalidation (Redis pub/sub, Kafka) or keep the TTL short.
6. Re-validate at decision points: checkout, payment release, permission-sensitive actions.
7. Monitor hit rate, latency, evictions, and staleness incidents.

Rule 6 is the golden rule restated as an operating procedure. A page can show a cached value; the action taken from that page re-reads the truth.

Freshness and size: TTL and eviction

Two separate mechanisms remove entries from a cache, and interviewers like to hear them kept apart. Expiry removes an entry because it is too old to trust. Eviction removes an entry because the cache is full and something else needs the room.

TTL: the dial between hit rate and staleness

TTL is the most common freshness policy, and the one DNS and news sites run on. It is also a dial with a cost at both ends.

The last row is why the practical rules say "TTL and evict on write". Eviction does the work of freshness; TTL becomes the backstop for the bug or the race. You can then afford a long TTL, and a long TTL is what earns the hit rate.

Move the TTL slider on the chart below and watch hit rate and stale reads move together. Then switch on evict-on-write and watch the stale-read curve flatten while the hit rate keeps its gains.

Add jitter to every TTL. If ten thousand keys are loaded at 09:00 with a TTL of one hour, they all expire at 10:00 and the database takes ten thousand misses at once. A TTL of 600 seconds plus a random 0–120 spreads them out. That is the avalanche, and the fix is one line.

Eviction: what goes when the cache is full

The difference between LRU and FIFO only shows when a hot key is also an old one. Run the demo below with four slots, keep re-reading one key while new keys arrive, and watch FIFO throw out the key you are reading while LRU keeps it.

In practice you rarely implement either. Caffeine uses a frequency-aware variant of LRU and Redis offers LRU and LFU approximations; what you choose is the size bound and the TTL. In a design answer, though, "bounded by size with LRU eviction, plus a TTL for freshness" is the sentence.

The metrics that tell you if the bet is paying

• Hit rate. The bet, measured. A falling hit rate means the working set outgrew the cache, or keys are being invalidated faster than they are read.
• Latency of the cache itself. A slow cache is worse than no cache, because every miss now pays both.
• Eviction rate. High eviction with a falling hit rate means the cache is too small for the hot set (question 4 failing in production).

When the cache itself fails

A cache introduces failure modes that did not exist before it. Know each one by name, by cause and by fix.

Two of these lean on other sections. The Bloom filter that stops penetration and the consistent hashing that spreads hot keys across cache nodes are both in R10. The circuit breaker that stops a cache outage turning into a database outage is in R14.

The cache outage is the one that turns into a full outage.

A database tuned behind a 95% hit rate is sized for 5% of the traffic. Lose the cache and it receives twenty times its normal load in one second. That is why "the cache is optional, the app just gets slower" is only true if the fallback is rate-limited. Netflix replicates EVCache across zones for exactly this reason.

Implementing it in Java and Spring Boot

Spring Boot 3.x, Java 17+. The examples follow the order of the argument: declarative cache-aside first, then in-process and distributed caches, then the consistency fixes, then the failure-mode defences.

Dependencies

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-cache</artifactId></dependency>
<dependency><groupId>com.github.ben-manes.caffeine</groupId><artifactId>caffeine</artifactId></dependency>
<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-redis</artifactId></dependency>

Declarative cache-aside with Spring's cache abstraction

@SpringBootApplication
@EnableCaching
public class App { public static void main(String[] a) { SpringApplication.run(App.class, a); } }

@Service
public class ProductService {
    private final ProductRepository repo;
    public ProductService(ProductRepository repo) { this.repo = repo; }

    // cache-aside read: miss → method runs → result stored
    @Cacheable(cacheNames = "products", key = "#id", sync = true, unless = "#result == null")
    public ProductDto get(long id) {
        return repo.findById(id).map(ProductDto::from).orElse(null);
    }

    // write DB, then evict (write-around + evict)
    @Transactional
    @CacheEvict(cacheNames = "products", key = "#cmd.id()")
    public void updatePrice(UpdatePrice cmd) {
        repo.updatePrice(cmd.id(), cmd.price());
    }

    // write-through style: return value is written into the cache
    @CachePut(cacheNames = "products", key = "#result.id()")
    public ProductDto create(NewProduct cmd) { return ProductDto.from(repo.save(cmd.toEntity())); }
}

Four things in this code bite people:

• sync = true is per JVM, not cluster-wide. Concurrent misses for one key wait for a single loader inside one instance; ten instances still send ten loads.
• Cache DTOs, not JPA entities. Lazy proxies and large object graphs do not serialise well.
• Self-invocation bypasses the proxy. this.get(id) from inside the same bean skips the cache entirely. Call through another bean.
• @CacheEvict and @Transactional on the same method do not guarantee order. The evict can run before the commit, and a reader can refill the old value in between. That is the "delete cache, then update DB" row from the consistency table, arriving through an annotation. Fix it with transactionAware() or an after-commit event, both below.

In-process cache: Caffeine for reference data and L1

@Configuration
public class CaffeineConfig {
    @Bean
    public CacheManager caffeineCacheManager() {
        CaffeineCacheManager mgr = new CaffeineCacheManager("bicDirectory", "holidayCalendar");
        mgr.setCaffeine(Caffeine.newBuilder()
                .maximumSize(50_000)
                .expireAfterWrite(Duration.ofHours(12))   // upper bound on staleness
                .recordStats());                          // hit/miss metrics via Actuator/Micrometer
        return mgr;
    }
}

Or with properties only:

spring:
  cache:
    type: caffeine
    cache-names: bicDirectory,holidayCalendar
    caffeine:
      spec: maximumSize=50000,expireAfterWrite=12h,recordStats

Programmatic read-through with single-flight per key:

LoadingCache<String, FxRate> fx = Caffeine.newBuilder()
        .maximumSize(1_000)
        .expireAfterWrite(Duration.ofSeconds(30))
        .build(pair -> fxProvider.fetch(pair));   // loader called once per key even under concurrency
FxRate r = fx.get("USDINR");

Refresh-ahead for hot reference data, so no payment ever waits on a reload:

LoadingCache<String, Calendar> holidays = Caffeine.newBuilder()
        .refreshAfterWrite(Duration.ofHours(1))   // after 1h, next read triggers async reload; old value served meanwhile
        .expireAfterWrite(Duration.ofHours(24))   // hard limit if reloads keep failing
        .build(ccy -> calendarService.load(ccy));

The pairing of refreshAfterWrite with expireAfterWrite is the point. Refresh keeps the value current under normal conditions; expiry is the hard ceiling if the calendar service is down for a day, so a stale calendar cannot live forever.

Distributed cache: Redis with per-cache TTLs

spring:
  data:
    redis:
      host: localhost
      port: 6379
  cache:
    type: redis

@Configuration
public class RedisCacheConfig {
    @Bean
    public RedisCacheManager cacheManager(RedisConnectionFactory cf) {
        RedisCacheConfiguration base = RedisCacheConfiguration.defaultCacheConfig()
                .entryTtl(Duration.ofMinutes(10))
                .disableCachingNullValues()
                .prefixCacheNameWith("svc-catalog:")
                .serializeValuesWith(RedisSerializationContext.SerializationPair
                        .fromSerializer(new GenericJackson2JsonRedisSerializer()));
        return RedisCacheManager.builder(cf)
                .cacheDefaults(base)
                .withInitialCacheConfigurations(Map.of(
                        "products",     base.entryTtl(Duration.ofMinutes(15)),
                        "userProfiles", base.entryTtl(Duration.ofMinutes(5)),
                        "fxQuotes",     base.entryTtl(Duration.ofSeconds(30))))
                .transactionAware()   // defers put/evict until the surrounding transaction commits
                .build();
    }
}

transactionAware() wraps the caches so that @CachePut and @CacheEvict inside a Spring-managed transaction apply after commit. Note the per-cache TTLs: each one is the data class from the classification table, written down as configuration. fxQuotes at 30 seconds stands in for whatever validity window the provider actually honours.

Evict after commit explicitly

This version works with any cache and any transport:

public record ProductChanged(long id) {}

@Service
class ProductWriter {
    private final ProductRepository repo; private final ApplicationEventPublisher events;
    ProductWriter(ProductRepository repo, ApplicationEventPublisher events) { this.repo = repo; this.events = events; }

    @Transactional
    public void updatePrice(long id, BigDecimal price) {
        repo.updatePrice(id, price);
        events.publishEvent(new ProductChanged(id));     // delivered only if the tx commits
    }
}

@Component
class ProductCacheInvalidator {
    private final CacheManager caches;
    ProductCacheInvalidator(CacheManager caches) { this.caches = caches; }

    @TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
    public void on(ProductChanged e) {
        Objects.requireNonNull(caches.getCache("products")).evict(e.id());
    }
}

The part people miss: this listener runs in memory after the transaction closes. R4 makes the point about publishing events, and it applies to evictions too: if the pod dies between commit and listener, the eviction never happens. For a cache that is tolerable, because the TTL eventually cleans up, which is exactly why rule 1 says always set one. For multi-service systems where a missed eviction is not tolerable, replace the in-process event with CDC (Debezium → Kafka → invalidator) or a transactional outbox, so the eviction survives crashes and retries.

Manual cache-aside with jittered TTL, null caching and a stampede lock

When the annotations are not enough, this is what they are hiding. Three failure-mode defences in one method:

@Service
public class ProfileCache {
    private static final String NULL = "__NULL__";
    private final StringRedisTemplate redis; private final ObjectMapper json; private final ProfileRepository repo;

    public ProfileCache(StringRedisTemplate redis, ObjectMapper json, ProfileRepository repo) {
        this.redis = redis; this.json = json; this.repo = repo;
    }

    public Optional<Profile> get(String userId) throws Exception {
        String key = "profile:" + userId;
        String cached = redis.opsForValue().get(key);
        if (cached != null) return NULL.equals(cached) ? Optional.empty() : Optional.of(json.readValue(cached, Profile.class));

        String lockKey = key + ":lock", token = UUID.randomUUID().toString();
        boolean leader = Boolean.TRUE.equals(redis.opsForValue().setIfAbsent(lockKey, token, Duration.ofSeconds(5)));
        if (!leader) {                       // someone else is rebuilding: brief wait, then retry cache
            Thread.sleep(50);
            String again = redis.opsForValue().get(key);
            if (again != null) return NULL.equals(again) ? Optional.empty() : Optional.of(json.readValue(again, Profile.class));
        }
        try {
            Optional<Profile> fromDb = repo.findById(userId);
            String value = fromDb.isPresent() ? json.writeValueAsString(fromDb.get()) : NULL;
            Duration ttl = fromDb.isPresent()
                    ? Duration.ofSeconds(600 + ThreadLocalRandom.current().nextInt(120))   // jitter vs avalanche
                    : Duration.ofSeconds(60);                                              // short negative cache
            redis.opsForValue().set(key, value, ttl);
            return fromDb;
        } finally {
            if (leader && token.equals(redis.opsForValue().get(lockKey))) redis.delete(lockKey); // (use a Lua script for atomic check-and-delete in production)
        }
    }

    public void evict(String userId) { redis.delete("profile:" + userId); }
}

The comment on the last line matters: "get, compare, delete" is three round trips and not atomic. A Lua script does it in one step. R8 covers why a lock with a timeout needs a token. Here the stakes are low: if the lock misbehaves, two instances rebuild the same key, which wastes a query and harms nothing.

Two-level cache: Caffeine L1 plus Redis L2, with pub/sub invalidation

@Component
public class TwoLevelCache {
    private final Cache<String, String> l1 = Caffeine.newBuilder().maximumSize(10_000).expireAfterWrite(Duration.ofSeconds(30)).build();
    private final StringRedisTemplate redis;
    private static final String CHANNEL = "cache-invalidate";

    public TwoLevelCache(StringRedisTemplate redis, RedisMessageListenerContainer container) {
        this.redis = redis;
        container.addMessageListener((msg, pattern) -> l1.invalidate(new String(msg.getBody())), new ChannelTopic(CHANNEL));
    }

    public String get(String key, Supplier<String> loader) {
        return l1.get(key, k -> {
            String v = redis.opsForValue().get(k);
            if (v == null) { v = loader.get(); redis.opsForValue().set(k, v, Duration.ofMinutes(10)); }
            return v;
        });
    }

    public void evict(String key) {
        redis.delete(key);
        redis.convertAndSend(CHANNEL, key);   // every instance drops its L1 copy
    }
}
// plus: @Bean RedisMessageListenerContainer container(RedisConnectionFactory cf) { var c = new RedisMessageListenerContainer(); c.setConnectionFactory(cf); return c; }

This is the Stack Overflow design in thirty lines. Keep the L1 TTL short: Redis pub/sub is fire-and-forget, so an instance that misses a message keeps its stale copy until the L1 TTL expires. The 30 seconds is the bound on that.

Write-behind for counters (sketch)

public void like(long postId) { redis.opsForValue().increment("likes:" + postId); redis.opsForSet().add("likes:dirty", String.valueOf(postId)); }

@Scheduled(fixedDelay = 5_000)
public void flush() {
    Set<String> dirty = redis.opsForSet().pop("likes:dirty", 500);
    if (dirty == null) return;
    for (String id : dirty) {
        String v = redis.opsForValue().get("likes:" + id);
        if (v != null) postRepo.setLikeCount(Long.parseLong(id), Long.parseLong(v));   // idempotent absolute write
    }
}

The flush writes an absolute value, not an increment. SET count = X gives the same result however many times it runs, so a retried flush is harmless. That is the naturally idempotent rung of R3's ladder. An increment-based flush would double-count on every retry. The price you accept, stated plainly: a Redis crash can lose up to one flush interval of likes. For likes, fine. For anything with a currency attached, not.

Observability and operations checklist

• Actuator: /actuator/caches and /actuator/metrics/cache.gets?tag=result:hit. Caffeine needs recordStats for these to report anything.
• Alert on hit-rate drops, Redis latency, evictions and memory (R15 covers turning these into alerts that mean something).
• Implement a CacheErrorHandler so a Redis outage degrades to database reads instead of 500s, and protect the database behind it with a rate limiter or circuit breaker.
• Warm critical caches on deploy. Load reference data at startup so a fresh pod does not stampede the database on its first payment.
• Namespace and version your keys. svc:v2:product:42 lets you bust everything at once by bumping the version, which is also how you deploy a change to the cached value's shape.

The whole section on one page

DECIDE   reuse high? miss expensive? staleness OK? hot set fits?   -> cache
PLACE    reference -> Caffeine | shared -> Redis | very hot -> L1+L2 | public HTTP -> CDN
FILL     cache-aside (default) | read-through | refresh-ahead (hot, predictable)
WRITE    update DB -> delete key (after commit) | write-through (read-after-write) | write-behind (counters only)
FRESH    always TTL (+jitter) | evict on write | CDC for multi-service | version keys
EVICT    size bound + LRU for the hot set | FIFO when value decays with age | TTL for freshness
PROTECT  sync/single-flight, null caching, Bloom filter, L1 for hot keys, fallback on cache failure
NEVER    cache reads that decide (balances, final stock, payment release, fresh permission revokes)

What to say in the interview

"A cache is a bet that data will be read again before it changes, so I only make it when reuse is high, the miss is expensive and brief staleness is harmless, and I cache reads that inform, never reads that decide. In a payment system that means the BIC directory, holiday calendars and cut-offs live in-process with refresh-ahead, FX quotes live in Redis for exactly their validity window, and the balance and limit checks that release money always go to the system of record. The default pattern is cache-aside with update-the-database-then-delete-the-key after commit, a jittered TTL as the safety net, and CDC-driven invalidation once more than one service writes the data. And I treat the cache and the database as two systems with no shared transaction, so I name the staleness bound I'm offering instead of calling it consistent."