In-memory cache

InMemoryCache is the default Cache implementation. It’s a Map with three things layered on top: LRU eviction bounded by maxEntries, lazy per-entry TTL, and an optional background sweep that reclaims expired-but-untouched entries. In-process, zero dependencies, lost on restart.

import { InMemoryCache, InMemoryCacheOptions } from 'actor-ts/cache';

// Defaults: maxEntries 10 000, cleanupMs 60 000.
const cache = new InMemoryCache();

// Or tune it with the builder:
const cacheOptions = InMemoryCacheOptions.create()
  .withMaxEntries(50_000)
  .withCleanupMs(30_000);
const tuned = new InMemoryCache(cacheOptions);

A plain object is the shorthand alternative — new InMemoryCache({ maxEntries: 50_000, cleanupMs: 30_000 }). A TTL is still supplied per call (see TTL handling), not at construction time.

Options

Field	Builder	Default	Meaning
maxEntries	withMaxEntries(n)	10000	LRU cap on stored entries. Infinity = unbounded.
cleanupMs	withCleanupMs(ms)	60000	Background expired-entry sweep interval (ms). 0 / Infinity disables the sweep.
prefixQuotas	withPrefixQuotas(table)	(none)	Per-key-prefix reservations, { 'rsp:': 8000, 'idem:': 2000 }. Unset, the map is undivided — see dividing one instance by key prefix.

Values are validated once, at construction, on the merged settings: maxEntries must be a positive integer (or Infinity), cleanupMs a non-negative number (or Infinity), and every quota in prefixQuotas a positive integer under a non-empty prefix, with the quotas summing to at most maxEntries. A bad value throws OptionsError — the builder, a plain object, and HOCON all hit the same check.

When to use it

Three scenarios:

1. Tests — fast, no IO, clean teardown via close().
2. Single-process production — one process, no need to share cache state across pods.
3. Dev / local — the same code without Redis on the laptop.

For multi-process deployments, use Redis or Memcached instead — each process would otherwise keep its own independent copy.

The operations

InMemoryCache implements the full Cache surface:

await cache.set('user:1', user, 60_000);            // value + optional TTL (ms)
const hit = await cache.get<User>('user:1');        // Option<User> — None on miss/expiry

await cache.setIfAbsent('lock:job', '1', 30_000);   // true iff it was stored
await cache.incr('ratelimit:1.2.3.4', 60_000);      // atomic ++, returns the new count

const many = await cache.mget<User>(['user:1', 'user:2']);  // Map<string, User>
await cache.mset(new Map([['a', user1], ['b', user2]]), 60_000);

await cache.delete('user:1', 'user:2');             // one or many keys
await cache.close();                                // stops the sweep + clears the Map

setIfAbsent is atomic here for a structural reason rather than a protocol one: its read and write sit in the same synchronous block, and the single-threaded event loop cannot interleave another caller between them. That holds within one process only — two Node processes each have their own Map, so a lock:job key guards nothing across them. For cross-process locking, use acquireLock over RedisCache.

get returns an Option<V> — None on a miss or after expiry.

TTL handling

await cache.set('key', value, 60_000);   // expires at now + 60s
await cache.set('key', value);           // no TTL — lives until evicted/deleted

Expiry has two paths. It is lazy on access — an entry’s deadline is checked on every get, mget, incr, and setIfAbsent, and an expired entry is dropped at that point. A background sweep then runs every cleanupMs (default 60 s) to reclaim expired entries that are never touched again, so they don’t sit in the Map waiting for a read. Set cleanupMs to 0 / Infinity to turn the sweep off and rely on lazy expiry alone.

incr sets the TTL only when it creates the counter (the value becomes 1); later increments don’t refresh it — the right semantics for a fixed-window rate limiter.

Bounded by default (LRU)

InMemoryCache is LRU-bounded at maxEntries (default 10 000): inserting a new key beyond the cap evicts the least-recently-used entry. This is what keeps a flood of distinct, never-re-read keys — attacker-chosen Idempotency-Key or rate-limit keys, for instance — from growing the map without limit.

Only a read counts as a use. get, incr and mget move a key to the most-recently-used end; set, mset and setIfAbsent do not. So a hot key survives, but an entry that is written once and never read back — an idempotency record still waiting for the client’s retry — ages towards eviction from the moment it is stored.

Recency is not the only criterion, though — it is the last of three. Eviction asks which key prefix an entry belongs to, then what it carries, and only then how recently it was read. Both of the earlier questions are the next section.

Set maxEntries: Infinity to opt out of eviction entirely. Only do this when you control the key space — an unbounded map OOMs the process eventually.

What eviction protects and what it does not

The map is kept in two halves, and the write that created an entry decides which one it lands in:

• Carries a guarantee — a setIfAbsent claim (a lock, an idempotency marker) or an incr counter (a rate-limit window), with a finite TTL. For these the cache is the source of truth, so losing one does not cost a round-trip; it voids the guarantee.
• Opportunistic — everything written by set / mset. These have a source of truth behind them, so losing one is a cache miss and the caller already handles it.

Eviction drains the opportunistic half first, least-recently-used end onwards. A flood of distinct response-cache keys therefore cannot push a live lock, another client’s rate-limit counter, or a stored idempotency record out of the map — which it could before, at the default configuration:

import { acquireLock, CacheExtensionId } from 'actor-ts/cache';

const cache = system.extension(CacheExtensionId).cache();   // maxEntries 10 000

const lock = await acquireLock(cache, 'lock:nightly-report', 60_000);
// ...then 10 000 distinct `cache.set(...)` writes through the same instance.
// The lock is still held: every one of those is an opportunistic write,
// and the opportunistic half is drained first.

A set picks a guarantee up in exactly one case: when it replaces a live claim under the same key. That is the shape idempotent uses — claim the key with setIfAbsent, then overwrite the marker with the finished response — and both halves of it have to be protected, or the record is exposed for the whole window the client’s retry lives in. Outside that case a set is never protected, whatever TTL it carries: every cached response body is a finite-TTL set, so protecting those would protect everything and therefore nothing.

maxEntries is still a hard cap, and that is the limit

Carrying a guarantee re-orders the victims; it never blocks an eviction. Once every entry in the map carries one, the least-recently-used of those goes — so the cap holds exactly as before, and a key flood still cannot grow the map. Which leaves four things to plan for:

• Size maxEntries above the number of claims, counters and locks live inside one TTL. A cache holding nothing but locks evicts its oldest lock at the cap, and the next caller then acquires a lock somebody still holds.
• The guarantee split does not rank guarantees against each other. Two guarantee-carrying consumers sharing one instance evict each other on recency alone — a counter flood takes idempotency records once the map holds nothing cheaper. Give each consumer its own cache (below), or divide the shared one with prefixQuotas (next section).
• A guarantee you store yourself with a plain set is not recognised — the cache cannot tell it from a response body. Write the claim with setIfAbsent, or take it with acquireLock.
• A claim with no TTL is not protected. An unbounded lock is the wedge setIfAbsent warns about, and protecting one would make it permanent — nothing would ever expire it.

And none of it follows a remote backend. Redis under maxmemory-policy allkeys-lru and Memcached both evict server-side, where no client-side policy reaches — see the Memcached page.

Dividing one instance by key prefix

prefixQuotas splits one map between the consumers writing into it. Each entry is a key prefix and the number of entries reserved for it:

import { InMemoryCache, InMemoryCacheOptions } from 'actor-ts/cache';

const cacheOptions = InMemoryCacheOptions.create()
  .withMaxEntries(10_000)
  .withPrefixQuotas({ 'rsp:': 7_000, 'idem:': 2_000, 'rl:': 1_000 });
const shared = new InMemoryCache(cacheOptions);

A quota is a cap and a reservation at once, and both halves are what make it a boundary rather than a hint:

• as a cap, a prefix that has reached its quota takes its next victim from inside itself — so a caller who can mint keys under one prefix evicts only that prefix’s own entries;
• as a reservation, the entries a prefix holds below its quota are not available to anybody else — so the flood cannot reach a rate-limit counter on the other side of the map either.

A key belongs to the longest configured prefix it starts with, and to a shared unreserved remainder when it starts with none. The quotas must sum to at most maxEntries: a reservation the map cannot honour is refused at construction rather than broken under load. Nothing about the guarantee split changes inside a bucket — opportunistic entries still go before guaranteed ones, least-recently-used first.

Two things it does not do:

• It bounds a prefix, not a caller. Two clients sending Idempotency-Keys share the idem: reservation and still evict each other. A per-caller key space (an identity scope over a known, small set of tenants, each reserved separately) is the only shape of this that helps, and Idempotency-Key on its own is not one.
• It does not grow the map. maxEntries remains the hard cap, so a configuration that reserves every slot leaves an unreserved write taking one from a reservation rather than the cap taking the loss.

There is no equivalent on RedisCache or MemcachedCache — both evict server-side, where no client-side policy reaches.

What a dropped guarantee costs

When the cap does reach one, nothing surfaces. The entry is simply gone, well inside its TTL:

• Rate limit — the counter disappears, so the next request from that client starts a fresh window at 1. The limit resets without anyone hitting it.
• Idempotency — the stored response disappears, so the client’s honest retry finds no record, re-claims the key and runs the handler a second time. That is a double charge on a payments endpoint.
• Lock — the entry a holder wrote disappears while the holder is still inside the critical section, so the next acquireLock on that key succeeds and hands the same lock out twice. The original holder’s release() then returns false, which reads like “the section overran its TTL” and here means the opposite: most of the TTL was left. Nothing in the return value separates the two causes.

One cache per consumer

The advice survives the policy, because the policy only re-orders victims inside one instance:

import { CacheExtensionId } from 'actor-ts/cache';

const extension = system.extension(CacheExtensionId);

const limiterCache     = extension.cache('rate-limit');
const idempotencyCache = extension.cache('idempotency');
const responseCache    = extension.cache('response-cache');

Each name resolves to its own instance, so a flood through one cannot reach the others at all. Size each one’s maxEntries for that consumer’s key space alone, under the cache’s own name:

actor-ts.cache {
  in-memory { maxEntries = 10000 }              # every in-memory instance

  idempotency.in-memory { maxEntries = 200000 } # just cache('idempotency')
  rate-limit.in-memory  { maxEntries = 50000 }  # just cache('rate-limit')
}

The per-name block wins over the global one leaf by leaf, so an override sets only what it names and the rest still falls through — the one exception is prefixQuotas, which is a table and is layered whole, because a half-inherited table is a sum nobody wrote down. The name is yours, so these paths are not listed in reference.conf — the same reason actor-ts.cache.<name>.plugin is not. A bad value is refused at the first cache(name) with an OptionsError rather than quietly sizing the map at the default.

Where one instance genuinely has to be shared — a single cache handed to all three middlewares — reserve each consumer’s share of it instead:

actor-ts.cache.shared.in-memory {
  maxEntries = 10000
  prefixQuotas { "rsp:" = 7000, "idem:" = 2000, "rl:" = 1000 }
}

The prefixes are the middlewares’ keyPrefix options, and they need quoting in HOCON because they contain a colon.

Either way the blast radius narrows rather than disappearing: a middleware whose own key space is attacker-controlled still evicts its own entries, and no quota changes that — a client with an IPv6 /64 mints rate-limit keys all day, and every one of them is a counter this policy protects under the same rl: reservation. Where that matters, back the security-relevant consumer with Redis (which you size and monitor independently) instead of an in-process LRU.

Sharing across the system

import { CacheExtensionId, InMemoryCache } from 'actor-ts/cache';

// The extension's `default` cache is an InMemoryCache out of the box:
const cache = system.extension(CacheExtensionId).cache();

// Override the default, or register a separate named cache:
system.extension(CacheExtensionId).setCache('default', new InMemoryCache());
const sessions = system.extension(CacheExtensionId).cache('sessions');

system.extension(CacheExtensionId).cache(name) resolves a cache by name — the default name is an InMemoryCache unless you replace it (via setCache, registerCache, or the HOCON path actor-ts.cache.<name>.plugin). HTTP middleware, projection actors, and your own code then share one configured instance instead of each building its own.

The in-memory cache built by the extension reads its defaults from HOCON:

actor-ts.cache.in-memory {
  maxEntries = 50000   # LRU cap
  cleanupMs  = 30000   # background sweep interval, ms (0 disables)
  # prefixQuotas { "rsp:" = 40000, "idem:" = 10000 }   # optional, off by default
}

That block applies to every in-memory instance; actor-ts.cache.<name>.in-memory overrides it for one of them — see one cache per consumer.

For a throwaway cache, just construct new InMemoryCache() directly — no extension needed.

When it’s wrong for production

Multi-process state divergence

// 4 pods, each with its own InMemoryCache → 4 independent caches

The same request hitting different pods can get a cached response from one and miss on another. Use Redis for shared multi-process state.

Restart loses everything

# kubectl delete pod my-app-1
# → the entire pod's cache is gone

In-memory caches don’t survive restart. For caches where the cold-start cost is significant (rebuilding takes minutes), use Redis (persists) or accept the cold start.

Tune maxEntries to your working set

// Default cap is 10 000 entries; raise it for a larger hot set…
new InMemoryCache({ maxEntries: 200_000 });
// …or opt out entirely — only if you control the key space.
new InMemoryCache({ maxEntries: Infinity });   // ✗ unbounded — OOMs eventually

The LRU cap protects you from unbounded growth, but a too-small cap evicts hot entries early. Size it to the working set you expect.

Where to next

• Cache overview — the bigger picture.
• Redis cache — multi-process alternative.
• Memcached cache — alternative.
• CachedSnapshotStore — one consumer of the cache abstraction.