Distributed Cache System — System Design Interview Practice

Interview prompt

Design Design a distributed caching system like Redis or Memcached that provides fast data access. so users can Store and retrieve key-value pairs reliably at scale.

• Define the source of truth for Store and retrieve key-value pairs; Distribute data across multiple nodes and make retries idempotent.
• Use bounded, partitioned state to meet Handle millions of requests per second and Sub-millisecond response times.
• Separate the critical request path from Consistent hashing for data distribution, Replication for fault tolerance, Memory management and eviction policies.
• Explain consistency, failure recovery, authorization, observability, and a degraded mode.

Requirements and scale assumptions

• Support the core workflow to Store and retrieve key-value pairs.
• Expose status, results, and freshness appropriate to Design a distributed caching system like Redis or Memcached that provides fast data access..
• Support authorization, validation, updates, deletion, and recovery semantics.
• Meet Sub-millisecond response times under normal load.
• Scale to Handle millions of requests per second without a single hot key or unbounded synchronous work.
• Do not lose committed state; make retries and duplicate events safe.
• Degrade safely when downstream workers, caches, or external dependencies fail.
• Handle millions of requests per second
• Partition by the primary tenant, user, item, or geographic key and isolate hot partitions.
• Keep serving state bounded; retain raw events or durable records for replay and auditing.

• Peak scale: Handle millions of requests per second — Capacity assumption that drives partitioning and backpressure.
• Latency target: Sub-millisecond response times — User-facing budget for the primary request or read path.
• Durable boundary: Committed before async — The source of truth is Store and retrieve key-value pairs; Distribute data across multiple nodes.
• Async boundary: At-least-once workers — Keep Consistent hashing for data distribution, Replication for fault tolerance, Memory management and eviction policies off the synchronous path.

Key entities

• ResourceSpecresourceId, tenantId, desiredState, version, policyVersion, updatedAt

  Versioned desired state for a distributed cache system managed resource.

• OperationoperationId, resourceId, requestHash, step, attempt, status

  Durable distributed cache system reconciliation operation with per-step progress.

• PolicyVersionpolicyId, scope, version, rules, effectiveAt, status

  Auditable distributed cache system policy evaluated before provisioning or mutation.

• ReconciliationCheckpointresourceId, provider, observedVersion, cursor, lastError, updatedAt

  Provider-specific distributed cache system observation and recovery cursor.

Data flow

1. 1. Accept a desired-state commandThe distributed cache system control plane authenticates the tenant, validates policy and quotas, checks the expected version, and records the desired state.
2. 2. Plan a safe operationA planner turns distributed cache system desired state into ordered, bounded steps with dependency checks, blast-radius limits, and rollback metadata.
3. 3. Reconcile providers asynchronouslyWorkers apply distributed cache system operations through provider adapters, persist checkpoints, rate-limit calls, and treat unknown outcomes as observable state.
4. 4. Publish observed healthThe serving projection joins desired and observed distributed cache system state with operation status, policy version, freshness, and actionable errors.
5. 5. Recover and auditRetries, dead letters, drift detection, and operator approvals repair distributed cache system resources without losing the original command or provider evidence.

Deep dives and trade-offs

• Desired versus observed stateKeep distributed cache system desired state separate from provider-observed state and show both to operators. Make every reconciliation step conditional and resumable so a worker crash does not restart unsafe effects. Version policy and resource state so old operations cannot overwrite newer intent.
• Provider failures and unknown outcomesUse provider-specific idempotency tokens and query-after-timeout behavior for distributed cache system operations. Bound retries with exponential backoff, circuit breakers, and per-provider quotas. Route irreconcilable drift to an approval or quarantine path instead of retrying forever.
• Blast radius and operationsPartition distributed cache system work by tenant, region, cluster, or resource class and cap concurrent mutations. Audit who changed desired state, which policy allowed it, and what provider evidence was observed. Alert on drift age, operation backlog, failed steps, policy denials, and stale observations.
• Push versus pull reconciliationUse event triggers for fast response and periodic scans for missed events, drift, and recovery. A push-only distributed cache system controller silently misses changes when a provider event is lost.
• Central control plane versus provider-native controllersKeep policy, intent, and audit centralized while isolating provider-specific application logic behind adapters. A monolithic controller becomes hard to scale and couples unrelated provider failure domains.
• Automatic repair versus approvalAutomate low-risk, reversible distributed cache system changes and require approval for destructive or high-blast-radius operations. Full automation without policy or blast-radius controls can turn a transient signal into a widespread outage.