Quorum reads and writes

DistributedData’s default update / get are local-only — they apply to the local replica immediately; propagation happens via gossip. For workloads needing stronger guarantees, the updateAsync and getAsync variants wait for explicit replica acknowledgments.

Local — the default

dd.update<GCounter>(
  'hits',
  GCounter.empty,
  (c) => c.increment(dd.selfReplicaId(), 1),
);

const counter = dd.get<GCounter>('hits');

These operations:

• update — apply locally, fire-and-forget. Returns immediately.
• get — read the local replica’s view.

Behind the scenes, gossip propagates updates to other replicas over the next few rounds.

This is the right default — 99 % of DD operations use these.

When local isn’t enough

Three common cases:

1. Read-after-write across nodes. Node A writes, node B immediately reads — the write might not have gossiped yet, so node B sees stale data.
2. Confirmed write. The app wants to know “at least N other replicas have my update before I move on” — e.g., before replying to a client.
3. Strongest-known read. When stale is unacceptable for this read (a balance check before a payment), force the read to consult majority replicas.

updateAsync — wait for write acknowledgments

await dd.updateAsync<GCounter>(
  'hits',
  GCounter.empty,
  (c) => c.increment(dd.selfReplicaId(), 1),
  { consistency: 'majority', timeoutMs: 2_000 },
);

Applies locally + sends to the gossip layer, then waits for acknowledgments from the configured number of replicas.

consistency	What it waits for
'local' (default)	Self only — local apply is the ack.
'majority'	⌊N/2⌋ + 1 replicas have acked.
'all'	Every up-member replica has acked.
{ from: 3 }	from replicas have acked (clamped to [1, N]).

The Promise:

• Resolves when enough acks arrive within timeoutMs.
• Rejects with a timeout error if not.

A timeout doesn’t undo the local write — the value is already applied locally and gossips normally. The rejection just signals “I’m not sure enough replicas saw it within the deadline.”

getAsync — read with consistency

const counter = await dd.getAsync<GCounter>('hits',
  { consistency: 'majority' });

Same consistency options. The replicator queries other replicas for their views, merges the responses, and returns the merged value.

This means a majority read sees at least every write that majority-acknowledged — the latest “confirmed” value.

Picking a consistency level

'majority' is the sweet spot for “important” reads/writes — covers most failure scenarios at modest latency. 'all' guarantees consistency but fails if any replica is down (or slow), making it brittle.

Pragmatic patterns

Write majority + read local

// Write with majority — guarantees other replicas know
await dd.updateAsync('counter', ..., { consistency: 'majority' });

// Read locally — fast, and the value reflects (eventually) what
// other writers wrote with majority
const value = dd.get('counter');

This is the common production pattern. Writes pay the majority cost; reads are cheap. The cluster eventually converges.

Read majority before a critical decision

// Read majority — see the latest known value across the cluster
const balance = await dd.getAsync<PNCounter>('balance',
  { consistency: 'majority' });

if (balance.value() < 0) {
  // ... reject the transaction
}

For one-off reads where staleness matters, pay the cost for a strong read. Don’t make every read a majority read — it defeats the local-first design.

All-write for irrevocable changes

await dd.updateAsync('config-locked', ..., { consistency: 'all' });

Rare — when you need every replica to have seen the change before moving forward. Brittle (any down replica fails the operation), so reserve for truly-irrevocable state transitions.

Timeouts

await dd.updateAsync('x', ..., {
  consistency: 'majority',
  timeoutMs:   5_000,
});

Default timeout is a fixed 5 seconds (DEFAULT_ASK_TIMEOUT_MS), independent of the gossip interval. Override per call when:

• Stricter latency budget — set lower; reject early if consistency can’t be achieved.
• Looser — for cross-region clusters where gossip RTT is high, raise the timeout.

On timeout, the Promise rejects with an error you can catch:

try {
  await dd.updateAsync('x', ..., { consistency: 'majority', timeoutMs: 1_000 });
} catch (e) {
  // The write succeeded locally but didn't quorum in time.
  // Decide: retry, surface the error, accept the eventual-consistency outcome.
}

Bounding the pending set

Every quorum call that cannot resolve on the self-ack alone parks a record — a promise, a timer, the set of peers it is waiting on — until either enough replies arrive or the deadline passes. Two settings bound that set, in code or via actor-ts.distributed-data:

import { DistributedDataId, DistributedDataOptions } from 'actor-ts';

const distributedDataOptions = DistributedDataOptions.create()
  .withMaxPendingQuorumRequests(1_000)
  .withMaxQuorumTimeout(30_000);

const dd = system.extension(DistributedDataId).start(cluster, distributedDataOptions);

Option	Default	What it does
maxPendingQuorumRequests	1000	Most unsettled quorum requests at once. updateAsync and getAsync share the budget. 0 disables it.
maxQuorumTimeout	30000	Ceiling in ms on a call’s timeoutMs; a larger value is clamped down. 0 disables it.

Past the cap, the call rejects immediately with an error naming the setting — it does not queue and it does not wait:

try {
  await dd.updateAsync('x', ..., { consistency: 'majority' });
} catch (e) {
  // "DistributedData refused a quorum write on "x": 1000 quorum requests
  //  are already pending (max-pending-quorum-requests = 1000)"
}

As with a timeout, a refusal doesn’t undo the local write — the value is applied before the quorum starts and gossips normally.

What the cap actually bounds

Not the queue — the replicator’s mailbox is unbounded like every other actor’s, so no updateAsync envelope is ever discarded on its way in. What accumulates is the unsettled set: each pending request keeps a promise, a timer and a target set alive until its deadline passes. Without a cap, a partition that stops quorums from completing lets all three pile up and then expire together as a timeout storm. The cap converts that into immediate rejections you can catch at the call site, while the request is still attributable to the caller that made it.

Watching it in production

With metrics enabled, the replicator publishes four series — the label sets are fixed, so they cost a handful of series and no more:

Metric	Kind	Labels
distributed_data_quorum_pending	gauge	—
distributed_data_quorum_timeouts_total	counter	operation = write | read
distributed_data_quorum_rejected_total	counter	operation = write | read
distributed_data_dropped_values_total	counter	—

The pair worth alerting on is the last two. A rising ..._rejected_total means callers are outrunning the cap — either a runaway producer or a cap set too low for the workload. A rising ..._dropped_values_total means a peer is sending CRDT payloads this replica cannot decode, which is a broken or hostile peer rather than a tuning problem.

What quorum doesn’t guarantee

`'majority'` isn't linearizability

A majority write + a majority read don’t give you full linearizable semantics. Concurrent writes still merge per CRDT rules; majority just guarantees enough replicas are in the loop. For true linearizable shared state, use a singleton.

Quorum across an unstable cluster

// Cluster of 3 nodes; one is flapping unreachable
await dd.updateAsync('x', ..., { consistency: 'majority' });
// Sometimes succeeds (2 acks), sometimes fails (1 ack)

Quorum is at-the-moment. If the cluster’s up-member set changes mid-call, the quorum criterion may temporarily fail. Handle the resulting timeout — retry or accept.

Don't quorum every operation

// Every write is majority — heavy cost
for (const item of items) {
  await dd.updateAsync('counter', ..., { consistency: 'majority' });
}

Quorum operations cost gossip RTT (typically 1-2 s). For bulk writes, batch and quorum once at the end, or accept local writes and let gossip propagate.

Where to next

• Distributed data overview — the bigger picture.
• Replication — how local writes propagate to peers.
• Durable storage — for state surviving full-cluster restarts.
• Designing data — when DD’s eventual-consistency is the wrong fit.