Each test is 5 questions with varying difficulty.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.
Memcached is a high-performance, distributed memory object caching system designed primarily to speed up dynamic database-driven web applications by alleviating database load. In the modern 2026 systems engineering landscape, while Redis has captured mindshare due to its rich data structures and persistence guarantees, Memcached remains a foundational study in ultra-low latency, multi-threaded caching and deterministic memory allocation. Software and systems engineers at top-tier technology companies are routinely tested on Memcached architecture during infrastructure and systems design interviews to evaluate their understanding of lock contention mitigation, multithreaded network models via libevent, and fixed-size slab allocation strategies. Interviewers expect junior engineers to understand basic key-value operations, cache-aside patterns, and eviction mechanics, whereas senior and staff engineers must demonstrate deep mastery over slab fragmentation, chunk size growth factors, lock-per-slab isolation, and network concurrency tuning. This page provides a comprehensive breakdown of Memcached internals, architectural patterns, and production challenges to help you ace your next technical interview.
Understanding Memcached architecture matters because it represents the zenith of raw, unadorned in-memory performance and minimalist design philosophy in systems engineering. In production environments handling millions of requests per second, such as content delivery networks, massive ad-tech bidding platforms, and large-scale social networks, Memcached serves as an ultra-fast caching layer that eliminates expensive disk and network I/O operations. Its architecture is built around two defining pillars: a non-blocking network I/O engine powered by libevent and a deterministic memory allocator called the slab allocator, which completely bypasses standard operating system malloc/free fragmentation overheads.
In system design interviews, Memcached is frequently referenced as the classic benchmark for horizontal cache scaling without centralized coordination. Because Memcached nodes are completely stateless and do not communicate with one another, scaling out is an exercise in client-side consistent hashing rather than cluster-wide consensus protocols like Raft or Paxos. Strong candidates distinguish themselves by explaining how Memcached avoids the memory fragmentation that plagues traditional memory allocators, detailing how items are categorized into power-of-two growth factor slab classes. Weak candidates merely treat Memcached as a black-box dictionary, showing ignorance of slab exhaustion, item expiration lazy deletion, and thread-locking bottlenecks.
Furthermore, studying Memcached provides vital insight into concurrency models. Unlike single-threaded event loops used in early caching engines, Memcached utilizes a master-worker thread architecture where a dispatcher thread accepts incoming TCP connections and distributes socket file descriptors to worker threads using a round-robin algorithm over pipe-based notifications. Mastering these low-level synchronization primitives and concurrency patterns directly translates to better design decisions across all high-throughput network services.
Memcached operates on a shared-nothing, multi-threaded architecture driven by an event-driven networking model. The system is split into a main dispatcher thread and a pool of worker threads. The dispatcher thread listens on the network socket, accepts incoming client connections using libevent, and delegates the accepted socket file descriptors to worker threads via a round-robin mechanism over a local pipe. Each worker thread runs its own independent libevent loop, managing its assigned client connections, parsing text or binary protocol commands, interacting with the central hash table, and directly manipulating the slab allocator memory structures. Thread synchronization is handled via fine-grained mutexes protecting individual hash table buckets and slab classes, ensuring high throughput on multi-core hardware without global lock serialization.
Client Application (TCP)
↓
[Main Dispatcher Thread (Accepts Connections)]
↓ (Round-Robin via Pipe)
[Worker Thread Pool (Libevent / epoll)]
↓
[Hash Table Lookup & Bucket Lock Acquisition]
↓
[Slab Allocator (Chunk Allocation / LRU Eviction)]
↓
[Memory Storage (Slab Pages / Item Chunks)]The application checks Memcached first for a requested key. On a cache miss, the application queries the persistent database, populates Memcached with the retrieved result, and returns it to the client. This pattern is implemented in application code using client libraries like pymemcache or spymemcached, ensuring the cache only stores actively requested data rather than preloading unnecessary records.
Trade-offs: Reduces memory waste and handles cache node failures gracefully, but introduces cache miss latency spikes and requires robust cache invalidation or TTL strategies to prevent stale data.
Clients or proxy routers like mcrouter map both Memcached cluster nodes and item keys onto an imaginary 32-bit integer circle using cryptographic hash functions like MurmurHash3 or MD5. When an item needs to be stored or retrieved, its key is hashed, and the router walks clockwise along the ring to locate the first available node. Virtual nodes are distributed across the ring to ensure uniform key distribution.
Trade-offs: Minimizes key remapping and cache invalidation storms when scaling nodes up or down, but requires careful handling of node failures and hot-spotting on heavily queried keys.
To prevent race conditions during concurrent updates, Memcached attaches a unique 64-bit transaction identifier (cas_uniq) to every item. When an application fetches an item for modification, it records the CAS token. Before writing the updated value back using the 'cas' command, Memcached verifies that the token has not changed. If another client modified the item in the interim, the operation fails, and the application retries.
Trade-offs: Provides lightweight, lock-free concurrency control across distributed readers and writers without holding expensive distributed locks, but can cause high retry CPU overhead under extreme write contention on hot keys.
| Reliability | Memcached is an in-memory cache with no built-in persistence, replication, or consensus protocols. Reliability in production is achieved through client-side redundancy, where applications write critical data to multiple independent Memcached nodes or rely on external replication proxies like mcrouter. If a node fails, clients experience a cache miss and fall back to the primary database, requiring database connection pooling and rate limiting to prevent thundering herd outages. |
| Scalability | Horizontal scalability is virtually linear because Memcached nodes share no state and do not communicate with one another. Cluster scaling is managed entirely by clients or proxy routers using consistent hashing rings. Adding or removing nodes only remaps a fraction of keys (1/N), minimizing cache invalidation overhead. |
| Performance | Memcached delivers sub-millisecond latencies (typically 0.2ms to 0.8ms P99) capable of handling hundreds of thousands of operations per second per node. Performance bottlenecks are typically limited by network card bandwidth (NIC saturation) or CPU context switching under high connection rates rather than internal algorithmic complexity. |
| Cost | Cost is driven almost entirely by RAM capacity and network bandwidth. Because data resides strictly in memory, caching massive datasets requires large RAM instances (e.g., AWS r6g memory-optimized EC2 instances). Cost optimization involves tuning slab growth factors to eliminate internal fragmentation and compressing payloads. |
| Security | Memcached traditionally lacks built-in authentication or encryption, relying on trusted internal VPC networks, security groups, and binding to private IP addresses. Modern deployments implement SASL (Simple Authentication and Security Layer) authentication and TLS termination proxies to secure data in transit across untrusted network boundaries. |
| Monitoring | Key metrics include global cache hit rate (target > 95%), eviction count per slab class, curr_items, bytes used vs limit_maxbytes, get_hits vs get_misses, and thread utilization. Prometheus scrapers combined with Grafana dashboards provide real-time visibility into slab exhaustion and connection pressure. |
Traditional allocators like malloc manage arbitrary-sized memory chunks on a general heap, which inevitably causes external memory fragmentation over time in high-churn workloads. Memcached's slab allocator pre-allocates 1MB pages from the operating system and divides them into fixed-size item chunks grouped by slab classes. While this can introduce internal fragmentation if object sizes do not align perfectly with chunk sizes, it completely eliminates external fragmentation and guarantees O(1) allocation and deallocation times without allocator lock contention.
Memcached was intentionally engineered around a minimalist philosophy: a high-performance, distributed key-value store optimized exclusively for raw speed and simplicity. Supporting nested data structures requires complex memory management, pointer overhead, and specialized command evaluation logic that can introduce latency jitter and compromise its sub-millisecond performance guarantees. Applications requiring complex data structures are expected to use Redis or relational databases, keeping Memcached focused on flat string key-value caching.
Memcached uses a master-worker thread model combined with fine-grained locking. A single dispatcher thread accepts incoming TCP connections and hands off socket descriptors to worker threads. Crucially, worker threads do not share a single global lock for memory operations. Instead, synchronization is partitioned across fine-grained mutexes protecting individual hash table buckets and slab classes. This ensures that threads modifying independent memory regions execute in parallel without serialization bottlenecks.
Mcrouter is an open-source memcached protocol router developed by Meta that acts as an intelligent intermediary proxy between application clients and backend Memcached clusters. It provides features such as connection pooling, request routing, automated sharding across hundreds of nodes, shadow traffic duplication, replication, and failover handling. By consolidating thousands of client connections into multiplexed backend pools, mcrouter protects Memcached nodes from connection exhaustion and simplifies cluster topology management.
When a slab class is full and a new item must be stored, Memcached enforces an LRU (Least Recently Used) eviction policy strictly within that specific slab class. It examines the tail item of the LRU queue for that slab class, reclaims its chunk, and assigns it to the new item. If eviction is explicitly disabled via startup flags (-M), Memcached refuses the storage operation and returns an out-of-memory error to the client rather than evicting existing data.
Internal fragmentation occurs when stored items are smaller than the fixed chunk size of the slab class they are assigned to, wasting the remaining bytes within that chunk. For example, if an item is 100 bytes and the assigned chunk size is 128 bytes, 28 bytes are wasted. Engineers mitigate this by analyzing slab stats and tuning the slab growth factor (-f) closer to 1.0 (e.g., 1.05 or 1.10) to create finer-grained chunk size steps, ensuring payloads match chunk sizes more closely.
Memcached nodes do not communicate, coordinate, or share state with one another across the cluster. There is no cluster-wide consensus protocol (like Raft or Paxos) or node discovery heartbeat mechanism running between Memcached instances. All clustering, sharding, and key routing logic is maintained entirely on the client side or via stateless routing proxies like mcrouter using consistent hashing rings, making individual Memcached nodes completely independent and easily replaceable.
CAS provides optimistic concurrency control without locking items across the network. Every item stored in Memcached is tagged with a unique 64-bit transaction identifier (cas_uniq). When a client retrieves an item, it retains this CAS token. When updating the item later, the client issues a 'cas' command passing the token back. Memcached verifies whether the item's token has changed in the interim. If another client modified it, the operation fails with 'EXISTS', prompting the application to re-fetch and retry.
While Memcached scales exceptionally well, primary bottlenecks at extreme scale include network card (NIC) bandwidth saturation, operating system kernel context switching overhead under massive connection churn, and fine-grained mutex lock contention on extremely hot keys (keys requested millions of times per second). Engineers address hot-key contention by implementing local application-level caching or read-aside replicas, and mitigate network limits by deploying high-performance 100Gbps network interfaces.
Lazy expiration means Memcached does not run active background cron threads to scan and purge expired keys across the entire memory space. Instead, when a client requests an expired key, Memcached checks the TTL timestamp, registers a cache miss, and immediately reclaims the chunk. While this avoids CPU overhead from background sweeping, it means that expired keys that are never requested again remain occupying memory chunks until those slab classes undergo LRU eviction when new items arrive.
Choose Memcached if your architecture requires ultra-fast, multithreaded, pure key-value caching with minimal operational overhead, predictable memory allocation via slabs, and maximum raw throughput on multi-core hardware. Choose Redis if your application requires advanced data structures (hashes, sorted sets, streams), built-in replication, persistence (RDB/AOF), Lua scripting, or pub/sub messaging capabilities, accepting the single-threaded event loop and memory management overhead that comes with it.
Memcached traditionally lacks built-in encryption (TLS) and authentication, making unencrypted instances exposed to the public internet vulnerable to reflection DDoS attacks and unauthorized data reading or flushing. Production deployments must bind Memcached to private VPC subnets, enforce strict security group firewall rules, use SASL authentication, and deploy TLS termination proxies (like stunnel or mcrouter) if traffic traverses untrusted network boundaries.
AI Prep covers AI Agents, Generative AI, ML Fundamentals, NLP & LLMs and a lot more, with adaptive tests and daily challenges. Fully offline on Android. Free to try, one-time unlock for lifetime access.