Stop Defaulting to LRU: How I Built a 15.5M ops/sec S3-FIFO Cache for Node.js by Hacking V8

Whenever we need an in-memory cache in Node.js, 99% of us do the exact same thing: npm install lru-cache. It’s muscle memory. LRU (Least Recently Used) has been the industry standard for decades, and for good reason—it’s intuitive and it works.
But if you are running a high-throughput backend, relying on LRU might be quietly bottlenecking your performance and degrading your cache hit ratios.
Recently, I went down a rabbit hole after reading the acclaimed 2023 SOSP paper: “FIFO queues are all you need for cache eviction”. The paper proves that a new algorithm called S3-FIFO (using three simple FIFO queues) completely outperforms complex LRU/LFU hybrids. It solves LRU’s biggest flaw: Cache Pollution caused by “one-hit wonders” (like a massive DB scan or web crawler traffic knocking out all your hot data).
High-performance storage engines and CDNs outside JavaScript have already adopted S3-FIFO. But the Node.js ecosystem was strangely quiet because we lacked a production-ready, ultra-optimized implementation.
So, I decided to build s3fifo. But I didn’t just want to port the algorithm—I wanted to push the V8 engine to its absolute limit.

Here is how I squeezed 15.5M ops/sec out of JavaScript, and why you should probably audit your cache packages today.

The Problem: High-Level Abstractions Hurt Performance

If you implement a cache in Node.js using naive JS Objects or Arrays, V8 engine overhead will destroy your throughput.
Typical LRU implementations allocate a { key, value, prev, next } node object on every single set() operation to manage a Doubly Linked List. Under high-throughput traffic, creating millions of transient wrapper objects triggers frequent, expensive Garbage Collection (GC) pauses.

To build a cache that outpaces industry defaults, I had to drop down to systems-level memory patterns within JavaScript.

Geeky V8 & Systems Optimizations

1. Zero-GC Flat Parallel Arrays & Slot Recycling

s3fifo allocates zero wrapper node objects at runtime. Instead, it uses Flat Parallel Arrays:

• #keys: (string | undefined)[]
• #values: (V | undefined)[]
• #meta: Uint32Array (Flags & Generation ID)
• #starts / #ttls: Float64Array
  Available slots are tracked and recycled using a contiguous Uint32Array stack (#freeSlots). Insertion and eviction are simple O(1) index shifts on flat arrays. Zero dynamic object allocations = Zero GC pressure.

// Slot index stack recycling: O(1) with zero GC
const slot = this.#freeSlots[this.#freeSlotTop--]!;
this.#keys[slot] = key;
this.#values[slot] = value;

2. Bit-Packed Metadata & Generation Tracking (32-bit Integers)

Instead of storing state flags inside JS objects, metadata for every slot is packed into a single 32-bit integer inside #meta (Uint32Array):

• Lower 5 bits: State flags (FREQ counter 0-3, RESIDENT, STALE, FREED)
• Upper 27 bits: Generation ID (increments every time a slot is recycled)

const META_FREQ_MSK  = 0b00011;
const META_STALE_MSK = 0b00100;
const META_RESI_MSK  = 0b01000;
const META_FREED_MSK = 0b10000;
// Read frequency counter via bitwise AND
const freq = meta & META_FREQ_MSK;

This packs all operational state into just 4 bytes per slot while staying immune to V8 hidden class shape transitions.

3. O(1) Zombie Detection via 64-bit Bit Packing

In S3-FIFO, the Ghost queue (#G) tracks items evicted from the main cache to give them a second chance if accessed again. But what happens if a slot is recycled for a new key while the old reference is still sitting in the Ghost queue (the ABA problem)?
Instead of using pointers or hashmap lookups, s3fifo packs the slot_index AND its generation_id into a single 64-bit float in the Float64Array Ghost ring buffer:

// Pack generation ID and slot index into one Float64
const packedForG = gen * SHIFT_ADDR + evictedSlot;
// Upon popping from Ghost queue:
const ghostOrZombie = packed % SHIFT_ADDR;
const poppedGen = Math.floor(packed / SHIFT_ADDR);
const currentGen = (this.#meta[ghostOrZombie]! >> 5) & GEN_MASK;
// If generations don't match, it's a "Zombie" (recycled slot) -> Discard in O(1)!
const isZombie = poppedGen !== currentGen;

4. Bitwise Operations Over Modulo Arithmetic

Ring buffers require index wrapping ((index + 1) % capacity). However, the modulo operator (%) is computationally expensive in hot execution loops.
s3fifo enforces internal ring buffer sizes to always be a power of 2. This allows replacing modulo operations with lightning-fast bitwise AND (&):

// Standard Ring Buffer (Slow Modulo)
this.tail = (this.tail + 1) % this.capacity;
// S3-FIFO Ring Buffer (Fast Bitwise Masking)
this.tail = (this.tail + 1) & this.mask;

5. Coarse-Grained Timestamp Caching for TTL

Calling performance.now() or Date.now() on every hit in a multi-million-ops loop adds measurable system call overhead. s3fifo maintains an internal #cacheNow timestamp updated via setInterval at a configurable resolution (ttlResolution, default 100ms) with .unref(), completely avoiding event loop blockage while eliminating syscall overhead.

📊 Benchmark: Hit Rate & Throughput vs lru-cache

Tested on a Zipfian distribution (skew 0.99, working set of 100,000 keys) comparing s3fifo v1.0 to Node’s popular lru-cache:

Average Hit Rate (%)

Average Throughput (ops/sec)

Production Ready & Keyv Integration

You don’t need to rewrite your application to try this out.

I’ve created keyv-s3fifo, which has been officially merged into the Keyv ecosystem as a first-class storage adapter. If you already use Keyv, you can switch your cache engine to S3-FIFO with a single line of code!

Conclusion

LRU isn’t dead, but defaulting to it without benchmarking is leaving major performance on the table. If your Node.js application handles large-scale traffic, web crawlers, or high-volume database queries, S3-FIFO will protect your hot data from cache pollution while delivering higher throughput.
Stop defaulting. Start benchmarking.
Check out the code, run the benchmarks on your machine, and if you find it useful, I’d love a ⭐️ on GitHub!
👉 GitHub: s3fifo

👉 npm: s3fifo

👉 npm: keyv-s3fifo