Pular para o conteúdo
Português (BR)

Dispatcher tuning

Este conteúdo não está disponível em sua língua ainda.

The dispatcher decides when actor messages run on the JavaScript event loop. Four shapes ship:

DispatcherSchedules viaFits
HybridDispatcher (default)queueMicrotask, every 64th turn setImmediateHTTP servers + mixed I/O; most apps.
MicrotaskDispatcherqueueMicrotaskMeasurement only — starves I/O under sustained load.
ImmediateDispatchersetImmediate / setTimeout(0)Every turn must yield to I/O.
ThroughputDispatchersetImmediate with N-then-yieldBatch processing across many actors.

The default — HybridDispatcher — is right for most apps. This page covers when it isn’t, and how to pick a better fit.

const system = ActorSystem.create('my-app');
// ↑ uses HybridDispatcher unless actor-ts.dispatcher.default says otherwise

Actor turns are handed over on the microtask queue, so a turn that follows another runs without waiting for the event loop. Every 64th turn goes through setImmediate instead, which lets I/O callbacks (HTTP handlers, broker messages, timer fires) run before the count starts over. A turn is up to actor-ts.actor.throughput messages (16 by default), so the loop gets a turn at least every ~1 024 messages.

For HTTP servers + broker-actor clusters (the common case), this keeps I/O on a bound without paying a setImmediate hop on every turn — the hop ImmediateDispatcher, the previous default, pays by design, at ~2.4 µs a turn that a request/response actor cannot amortise. actor-ts.dispatcher.default selects another dispatcher system-wide; an unset or unrecognised value means the hybrid.

Symptom: high HTTP latency under actor load

Section titled “Symptom: high HTTP latency under actor load”
P99 HTTP response time is 200ms; actors are processing tens
of thousands of messages/sec. The actor work isn't the
problem — it's that HTTP requests can't get a turn.

Cause: an actor (or group of actors) is processing messages so fast that HTTP handlers wait for a turn.

Fix: keep the default dispatcher and lower the busy actor’s own batch budget:

import { ActorOptions } from 'actor-ts';
const heavyOptions = ActorOptions.create().withThroughput(4);
const heavyActor = system.spawnAnonymous(HeavyWorker, heavyOptions);

Each of the heavy actor’s turns now ends after 4 messages instead of 16. The default dispatcher lets the event loop run every 64th turn, so the stretch of the heavy actor’s work HTTP can wait behind shrinks from ~1 024 messages to 256. HTTP latency drops; throughput on the heavy actor falls off only as far as the smaller batch costs it.

If HTTP still waits too long, yield more often system-wide: a smaller hybrid budget (Dispatchers.Hybrid(16) lets the loop run every 16th turn), or ImmediateDispatcher, which yields after every turn at the cost of the setImmediate hop the default exists to avoid. The budget counts turns, so it is an integer from 1 up, and at 1 the loop runs after every turn. 0, a negative or fractional count, NaN and Infinity are refused with an OptionsError when the dispatcher is built: below 1 no turn would ever run, and a budget that never runs out never yields.

Symptom: low actor throughput, low CPU usage

Section titled “Symptom: low actor throughput, low CPU usage”
The actor system is doing 1000 msg/sec on an idle CPU.
Profile shows time spent in setImmediate.

Cause: turns go through the event loop more often than the work needs. ImmediateDispatcher sends every turn through setImmediate by design; the default sends one turn in 64. For tight-loop CPU work without I/O, those yields are wasted.

Fix: if the system runs on ImmediateDispatcher — configured, or set per actor — go back to the default. If it already runs on the default, raise the hybrid’s budget so it yields less often:

import { ActorSystem, ActorSystemOptions, Dispatchers } from 'actor-ts';
const actorSystemOptions = ActorSystemOptions.create().withDispatcher(Dispatchers.Hybrid(1_000));
const system = ActorSystem.create('compute-pipeline', actorSystemOptions);

The event loop now gets a turn every 1 000 actor turns instead of every 64 — up to ~16 000 messages at the default batch. Caveat: that is the delay every timer and socket read in the process can see, so raise the budget only where nothing needs a prompt turn:

  • The system doesn’t serve HTTP traffic (compute-only workers).
  • Its actors don’t await I/O (purely CPU).

MicrotaskDispatcher is not the fix, even though it removes the yields entirely: with no budget at all, a busy system never lets timers or sockets run again — see Dispatchers. For the same reason Dispatchers.Hybrid(Infinity) is refused rather than taken as a hybrid that never yields.

new ThroughputDispatcher(100); // queued actor turns per tick

The constructor is positional — throughput first, an optional dispatcher id second:

  • throughput — queued turns drained per tick, across every actor sharing this dispatcher. Not messages, and not per actor: each turn belongs to a different actor, because an actor may only have one turn queued at a time. Higher = more throughput, worse I/O interleaving. Common values: 10-1000. Defaults to 16. An integer from 1 up: 0, a negative or fractional count, NaN and Infinity are refused with an OptionsError when the dispatcher is built, because below 1 no turn is ever drained and a drain without a bound never yields. The configured actor-ts.dispatcher.throughput takes the same values and is not raised to 1 the way the per-actor throughput is: with default = throughput in that block, a 0 there fails ActorSystem.create.
  • Between batches it always yields via setImmediate (falling back to setTimeout(0) where setImmediate is unavailable), so I/O and timers interleave. This isn’t configurable.

For a batch processor with many actors handling broker messages: throughput: 200 is a reasonable starting point.

How many messages one actor handles per turn before it yields is a separate setting from anything on the dispatcher:

import { ActorOptions } from 'actor-ts';
const bulkOptions = ActorOptions.create().withThroughput(64);
const bulk = system.spawnAnonymous(BulkProcessor, bulkOptions);

System-wide, the same knob is actor-ts.actor.throughput:

actor-ts {
actor {
throughput = 64
}
}

For one actor, or one family of them, whose spawn site you cannot edit, an actor-ts.deployment entry sets it per path:

actor-ts.deployment {
"/user/bulk" { throughput = 64 }
"/user/pool/routee-*" { throughput = 32 }
}

Precedence is the usual one, a layer longer — withThroughput() beats the matching deployment entry, which beats actor-ts.actor.throughput, which beats the built-in default of 16.

This is the setting that removes scheduling overhead for a busy actor. Every message used to cost a full setImmediate round trip (~2.4 µs) regardless of any dispatcher setting; batching amortises that across the whole batch, which is worth roughly 2-3× on a tell-driven workload.

The trade-off is fairness, and it is direct: a batch runs to its budget without yielding, so budget × handler time is the delay every timer, socket read and other actor can see. Lower it toward 1 for an actor with a slow handler that shares a system with latency-sensitive work; raise it for a short-handler actor that is a throughput bottleneck. A batch always ends early on an empty mailbox, a stop or suspend, and a throttle bucket that runs out — so the budget is a ceiling, never a commitment.

What a message costs when nothing is watching

Section titled “What a message costs when nothing is watching”

Worth knowing before you tune anything, because it sets the scale everything else is measured against.

On ImmediateDispatcher the scheduling round trip dominates everything else on the message path — about 2.4 µs against a raw mailbox operation of tens of nanoseconds. That is why, on that dispatcher, per-actor throughput is the setting with the largest effect on a tell-driven workload, and why a faster queue or a leaner handler moves the number far less than it feels like it should. The default HybridDispatcher exists to take that hop off most turns: it hands them over as microtasks and pays the setImmediate only once every 64.

Below that sits the framework’s own per-message overhead with metrics and tracing off. It used to be four extension-registry lookups, four metric label objects built for a registry that discards them, a closure, a throwaway array, and two clock reads — per message, on every system, whether or not anything was observing. #411 removed all of it; what remains is a handful of field reads and null checks. Both extensions are still free to switch on at runtime, and a message is either wholly instrumented or wholly not — the handles are resolved once per message rather than at each instrumentation point.

If you are chasing allocation pressure rather than latency, benchmarks/memory/receive-path.ts is the arm that isolates this, and its header explains why the effect shows up as throughput rather than as heap.

import { ActorOptions, Dispatchers } from 'actor-ts';
const computeOptions = ActorOptions.create().withDispatcher(Dispatchers.Hybrid(1_000));
const heavy = system.spawnAnonymous(BulkProcessor, computeOptions);
const httpHandler = system.spawn(
HttpHandler,
// → uses the system's default dispatcher (HybridDispatcher unless configured)
);

Mix freely: a compute-only actor can run on a hybrid with a raised budget — it yields to the event loop every 1 000 turns instead of every 64 — while HTTP handlers stay on the default. Not on a MicrotaskDispatcher, though: microtasks drain before any timer or socket callback whichever dispatcher queued them, so an unbounded chain on one actor starves the handlers beside it exactly as a system-wide one would.

What a per-actor dispatcher is not good for is batching. A ThroughputDispatcher given to one actor drains a queue that never holds more than that actor’s single pending turn, so its budget is unreachable — use withThroughput() instead. A ThroughputDispatcher earns its keep when a group of actors shares it.

const actorSystemOptions = ActorSystemOptions.create().withDispatcher(new ThroughputDispatcher(100));
const system = ActorSystem.create('my-app', actorSystemOptions);

Override the default for every actor that doesn’t specify otherwise. Useful for batch-only systems with no HTTP traffic.

Two stock metrics split a message’s latency at the moment it is picked up:

actor_mailbox_wait_seconds — queued, waiting its turn
actor_message_handler_seconds — inside onReceive

Read them together; the pair is what tells you whether tuning the dispatcher can help at all:

  • Wait p99 high, handler p99 low — the actor is fine and messages are sitting in the queue. Dispatcher tuning helps: raise the per-actor throughput budget so each turn drains more of the backlog, or move the actor off a dispatcher it is sharing with slow work.
  • Handler p99 high — the handler itself is the cost. No dispatcher setting makes it faster; that is a code or downstream problem, and raising the budget makes latency worse for everything sharing the runtime.

Queueing delay used to have to be inferred from the spread between p50 and p99 of the handler histogram, which conflated it with variance in the work itself. It is now measured directly.

The actor_mailbox_size gauge under load shows whether actors are keeping up. Persistently growing depth = either a slow handler or a misconfigured dispatcher. It is the later signal of the two — a series only exists above 10 000 queued messages, where wait starts climbing as soon as an actor falls behind. actor_mailbox_depth fills that gap: a label-free histogram observed once per delivery, so a burst that never reaches 10 000 still shows its tail.

Both metrics above are about an actor’s mailbox. A message can also wait one level up — in the dispatcher, after the cell has asked for a turn and before that turn starts:

actor_dispatcher_queue_delay_seconds{dispatcher="..."}

At rest this is a single scheduling hand-off, microseconds. When it climbs, the actor is not the problem and neither is its mailbox: turns are queued behind other turns, which is what dispatcher tuning is for.

histogram_quantile(0.99,
rate(actor_dispatcher_queue_delay_seconds_bucket[5m]))
# Per dispatcher. A high p99 on one and not the others tells you which
# pool to split, and which actors to move off it.

Reading it beside the mailbox pair completes the diagnosis:

  • Delay p99 high — turns are competing. Either something on this dispatcher is not yielding (a ThroughputDispatcher with too high a budget, or a handler that blocks), or too many actors share it. Split the dispatcher, or lower its throughput so it yields more often.
  • Delay p99 low, mailbox wait p99 high — the turns start promptly and one actor’s own queue is the backlog. Raise that actor’s withThroughput so each turn drains more, rather than touching the dispatcher.

There is deliberately no dispatcher_saturation_ratio — a 0–1 busy fraction cannot be computed honestly on all three supported runtimes, and Stock metrics records the measurements behind that decision. MicrotaskDispatcher is the one case where a low delay is not evidence of headroom, for the reason noted there.

HTTP server + actors → HybridDispatcher (default)
Compute-heavy + no HTTP → Dispatchers.Hybrid(1_000)
Batch processing, many actors → ThroughputDispatcher (throughput 100-500)
One actor is the bottleneck → default dispatcher + withThroughput(64...256)
Mixed: heavy actor + HTTP → default dispatcher + withThroughput(2...8) on the heavy actor
Every turn must yield to I/O → ImmediateDispatcher