Cluster overview
Это содержимое пока не доступно на вашем языке.
A cluster is a group of ActorSystems — typically one per node —
that know about each other. Nodes gossip their membership state,
detect each other’s failures, and route messages across the network.
Once joined, code on any node can tell an actor on any other node;
the cluster transport hides the wire.
Two ways to think about it:
- From the application’s perspective: one logical actor system
spread across N nodes. An
ActorRefcan point at an actor on any node; the same code that worked single-node still works. - From the runtime’s perspective: N independent systems exchanging gossip + heartbeats, each tracking who’s alive, routing messages through a chosen transport.
A minimal example
Section titled “A minimal example”Cluster.bootstrap packages the four steps you’d otherwise wire by
hand — create the ActorSystem, resolve seeds, Cluster.join,
hook up SIGTERM/SIGINT — into a single call:
import { Cluster, ClusterBootstrapOptions } from 'actor-ts/cluster';
// Local dev — no env vars, no seeds: single-node cluster.const { system, cluster, shutdown } = await Cluster.bootstrap(ClusterBootstrapOptions.create('my-app'));
// With explicit overrides:const { system, cluster } = await Cluster.bootstrap( ClusterBootstrapOptions.create('my-app') .withHost('0.0.0.0') .withPort(2552) .withSeeds(['node-a:2552', 'node-b:2552']) .withRoles(['compute']),);
// Once joined:cluster.upMembers(); // → ReadonlyArray<Member> of up-status nodescluster.subscribe((evt) => { /* MemberUp, MemberDown, ... */ });For full control (custom dispatcher, manual signal handling, own discovery loop) the lower-level pair still works:
import { ActorSystem } from 'actor-ts';import { Cluster, ClusterOptions } from 'actor-ts/cluster';
const system = ActorSystem.create('my-app');const clusterOptions = ClusterOptions.create() .withHost('0.0.0.0') .withPort(2552) .withSeeds(['node-a:2552', 'node-b:2552']);const cluster = await Cluster.join( system, clusterOptions,);A few settings are doing most of the work:
| Setting | Purpose |
|---|---|
host + port | The interface and port this node binds. 0.0.0.0 means “every interface” and is the default. |
advertisedHost + advertisedPort | The address peers dial — this node’s identity. Unset, the host is derived from host when that is routable, then from CLUSTER_HOST / POD_IP / HOSTNAME, then loopback; the port is simply port. |
seeds | Other nodes’ addresses. At startup, this node contacts them to join the existing cluster. An empty list = “I’m the first one” (auto-promotes to leader). |
roles | Tags this node carries. Routers and shard regions can filter on these (role: 'compute' skips nodes without the tag). |
appVersion | The application version this node states to its peers — a label for operators, shown on /cluster/members and wherever a member is printed, so the mixed window of a rolling update can be read off the cluster. Nothing places, routes or downs on it. Usually set by the deployment: actor-ts.cluster.app-version = ${?APP_VERSION} — quote a numeric-looking version inside the variable (APP_VERSION='"1.10"'): an environment value is read as JSON first, so an unquoted 1.10 reaches the peers as 1.1. |
After Cluster.join resolves, the node is in the cluster
(though possibly still in joining state for a few seconds until
convergence).
Reaching the cluster from an actor
Section titled “Reaching the cluster from an actor”Cluster.join also publishes the instance onto the ActorSystem, so
you do not have to thread the handle through every actor that needs it:
system.cluster; // Option<Cluster> — None if this system never joinedthis.context.cluster; // the same Option, inside an actorthis.cluster; // unwrapped, inside an actor — throws if there is noneThe unwrapped form is the one to use in code that only ever runs clustered, which includes every sharded entity and every singleton — those are constructed by the framework, so there is no call site to inject a cluster at:
class CartEntity extends Actor<CartMessage> { override preStart(): void { // No constructor argument, no closure, no options field. this.log.info(`cart ${this.entityId} on ${this.cluster.selfAddress}`); }
private onCheckout(): void { if (this.cluster.isLeader()) { /* ... */ } }}this.cluster.sharding and this.cluster.singleton come along with it,
so an actor can start a region or a singleton from the inside.
Reach for this.context.cluster instead when the actor must also run on
a plain, unclustered system — it answers None rather than throwing.
Both read through to system.cluster on each access, so an actor that
outlived the join, or a system that rejoined after leave(), always
sees the current instance.
The membership states
Section titled “The membership states”Every member goes through a small state machine:
joining— just announced itself. Other peers know about it but it isn’t routable yet.weakly-up(optional) — gossip-visible to peers but the leader hasn’t confirmed it. Useful for partition-tolerant joins; see Weakly-up.up— fully in the cluster, routable. This is the steady-state status.unreachable— the failure detector flagged this peer as not heartbeating. Still officially a member, but routing avoids it. Flips back toupwhen it heartbeats this node again — after a pause, though not after a symmetric partition, which leaves neither side heartbeating the other (issue #930). If it stays silent, the failure detector removes it atdown-after— the default with no strategy configured — or it stays here until something removes it — seedown.leaving— the member is gracefully leaving (viacluster.leave()). Its peers record it asremovedas soon as its leave reaches them.down— removed by a decision: a downing strategy’s, an operator’s (Cluster.down,POST /cluster/down), or — with no strategy configured, under the defaultfailure-detector.auto-down = on— the failure detector’s own. Passes straight on toremoved.removed— out of the cluster. A leave, a strategy’s decision and an operator’s down keep it as a tombstone for a TTL (default 24 h), so stale gossip from a slow peer can’t accidentally resurrect it. Theauto-downroute deletes the entry instead, which is why a peer that still lists the member can bring it back — see the failure detector.
The cluster’s events stream surfaces every transition — subscribe
to MemberUp / MemberRemoved / MemberUnreachable etc. and
react.
Subscribing, and what the replay tells you
Section titled “Subscribing, and what the replay tells you”cluster.subscribe(listener) starts by replaying the membership
that already exists, so a listener that attaches ten minutes into a
run still learns the world it joined instead of waiting for the next
change. Two forms, chosen per subscription:
import { CurrentClusterState } from 'actor-ts/cluster';
// 'events' (the default) — the membership as the events that built it:// MemberJoined per member, then the status event that member has reached,// then LeaderChanged. The live stream's own handler covers the replay.cluster.subscribe((evt) => { /* … */ });
// 'snapshot' — one CurrentClusterState, whatever the cluster's size.cluster.subscribe( (evt) => { if (evt instanceof CurrentClusterState) { console.log(`${evt.members.length} members, ${evt.unreachable.length} unreachable`); } }, { replayMode: 'snapshot' },);Pick 'snapshot' when the listener wants to know where things
stand rather than to re-live how they got there: it is one callback
instead of one per member, and it marks where the replay ends — which
the event form cannot. unreachable is a subset of members, not a
set beside it, and CurrentClusterState is never published on the
event stream: it describes one subscriber’s starting point, not
something that happened to the cluster.
The replay always reflects getMembers(): tombstones are excluded,
and every member is announced in the status it actually holds — an
unreachable peer replays as unreachable, not as a fresh join.
Live events follow three rules:
- An event goes to the listeners subscribed when it was emitted and still subscribed when their turn comes (#1294). A listener that subscribes from inside a callback starts with the next event — its replay already reflects the change in flight — and one whose unsubscribe has run is not called again, not even with the event in flight.
- Each status transition is announced once (#1428). Four paths
used to announce it twice — the failure detector’s
unreachableanddownverdicts, an applied downing decision andCluster.down— so two nodes could count one change differently. - Dispatch is synchronous and depth-first. A listener that
changes the membership inside its callback — downs a peer, leaves
the cluster — has the nested events delivered to every listener
before the outer event reaches the listeners subscribed after it.
With one listener that downs a peer on
MemberUnreachableand a second subscribed behind it, the second receivesReachabilityChanged,MemberDown,MemberRemovedand only thenMemberUnreachable, still carrying the statusunreachable.
When order matters, subscribe to the cluster’s events on
system.eventStream instead. The cluster publishes each event there
before it calls any listener, so an actor subscribed there receives
them in the order they were emitted, and a Cluster.subscribe
listener that changes the membership cannot reorder them.
A periodic sample, when you want one
Section titled “A periodic sample, when you want one”Every event above states a change. If what you want is “tell me
where the membership stands every N seconds”, set a cadence and
subscribe to ClusterStatsPublished:
actor-ts.cluster.publish-stats-interval = 30s # 0 (the default) arms no timerimport { ClusterStatsPublished } from 'actor-ts/cluster';
cluster.subscribe((evt) => { if (evt instanceof ClusterStatsPublished) { console.log(`${evt.up}/${evt.members} up, ${evt.unreachable} unreachable`); }});It carries the member count, the up count, the unreachable count,
the leader as an Option<Member> and this node’s own address — every
one of them already reachable through the Cluster API, which is the
point: what the key buys is the cadence, so a dashboard does not
have to run a polling loop of its own.
Three properties worth knowing:
- Node-local, like every other cluster event, and reached through
system.eventStreamas well ascluster.subscribe. It is deliberately not oncluster.eventStream: a per-node periodic sample on the cluster-wide bus costs N frames per node per interval to say what each node can already read locally. Bridge it yourself if you want a fleet-wide view —selfAddressis there so the samples stay distinguishable when you do. - Never replayed. A subscriber that attaches gets the membership replay described above and no stats sample: replaying a measurement taken before you were listening is just a stale number.
- Off by default.
0arms no timer at all, which is what every release before the key did.
It is not the only place these figures appear, and the three do not
answer quite the same question. The
DevTools dashboard samples them on
its own ticker and additionally counts nodes that recently left, so a
cluster of three that lost one still reads “2 / 3 up” — this event
counts the live member map instead. The cluster_members_up
metric is a pull surface with no
callback. This one is the framework-native push.
The leader
Section titled “The leader”cluster.leader() returns the lowest-addressed up member —
whichever member’s host:port sorts first. It is emphatically not
the oldest member: address order and join order are unrelated, so a
node that joins last leads immediately if its address sorts lowest, and
it takes over whatever the leader hosts (a cluster singleton, shard
allocation).
That is deliberate. The one property the leader has to have is that every node names the same one, and address order gives that from gossip every node already has — no monotonic join sequence has to travel on the wire. The cost is that “who leads” is decided by addressing rather than uptime: stable across a restart of the same pod, not stable across a re-address.
What gossip does
Section titled “What gossip does”Gossip is how members agree on who’s in the cluster. Every
gossipIntervalMs (default 1000 ms), each member picks a random
reachable peer and exchanges its view of the cluster. Over a few
rounds, every peer converges on the same state — without any
central coordinator.
The protocol carries:
- Membership table — every member’s address, status, roles, and version vector.
- Reachability observations — “I haven’t heard from X recently.”
Two peers exchanging gossip merge their tables by picking the higher version for each member. This is what makes convergence work without a leader-elected coordinator: every concurrent update is eventually seen by every peer.
For the deep dive, see Joining and seeds.
What the failure detector does
Section titled “What the failure detector does”The default failure detector is a simple, deterministic one based on elapsed-time thresholds. Every node sends each reachable member a heartbeat on a fixed cadence and tracks, per peer, when it last heard from it: any frame counts as a sign of life, and a peer that stays silent past a threshold is flagged. (The opt-in φ-accrual detector reads the same liveness, but learns its timing from heartbeat frames only — see liveness is any frame.)
unreachableAfterMs— once a peer has been silent this long, the member is markedunreachable.downAfterMs— if the silence persists this long, the detector’s verdict isdown. With no downing strategy that verdict removes the member —failure-detector.auto-downis on by default — and the surviving nodes take over its shards and singletons; under the opt-outauto-down = offit stays parked atunreachableuntil an operator or a strategy removes it — see Split-brain and downing.
Plain elapsed-time limits, no statistical variance tracking — sufficient for LAN-scale clusters. See Failure detector for tuning, the φ-accrual alternative and the WAN caveats.
Split-brain and downing
Section titled “Split-brain and downing”When the network partitions, two halves of the cluster may both remain operational but lose contact with each other. Without intervention, both sides keep running and accept conflicting writes — the classic split-brain problem.
actor-ts ships several downing strategies:
| Strategy | What it does |
|---|---|
KeepMajority | The side with more nodes wins; the smaller side downs itself. |
KeepOldest | The side containing the lowest-addressed member wins — “oldest” is address order, not join order. |
KeepReferee | A designated referee node’s view wins. |
StaticQuorum | The side that meets a configured quorum size wins; a side below it downs itself. |
LeaseMajority | The majority side wins, but only while it holds a coordination lease. |
See Downing strategies for
the full set. Pick deliberately. With no strategy — the default —
the failure detector removes an unreachable member itself once its
verdict is down (actor-ts.cluster.failure-detector.auto-down
ships on), and the last available node takes over the dead
member’s shards and singletons. That is right for a crashed node and
forks the cluster under a partition, because every side removes the
others at the same moment and both halves run on as two clusters
that do not merge again when the network heals
(issue #930).
So for production, configure lease-majority or another strategy.
With two or more survivors, one survivor’s gossip can bring a member
the other has just removed back as unreachable until it is removed
again — see
the failure detector.
auto-down = off is the opt-out: the detector parks the member at
unreachable, its shards stay homed on it and its remote watchers
get no Terminated, until an operator downs it (Cluster.down, or
POST /cluster/down with
actor-ts.management.enable-down-endpoint = on), a strategy is
configured, or it restarts on the same address. Neither setting
keeps the leader in place: the leader is the first up member in a
node’s view, so the members that can still reach each other elect
another one, and a singleton without a role moves with it.
Cross-node messaging
Section titled “Cross-node messaging”Once two nodes share a cluster, ref.tell(message) to a foreign-node
ref just works:
const remote = await system.actorSelection( 'actor-ts://my-app@10.0.0.5:2552/user/api/sessions/user-42',).resolveOne();
remote.tell({ kind: 'whatever' });// → serialized, sent over the transport, delivered to the foreign actor's mailboxThe cluster transport frames the message as a tagged JSON tree —
the same format the persistence stores write, so a Date, Map,
Set, bigint or Uint8Array in the body arrives as itself
(#450) — includes the routing path, and the destination’s transport
delivers it. Reply-to refs serialize cleanly — the receiving node
re-attaches a remote-routable handle so replies flow back over
the same transport.
See Refs across nodes for the wire-format details.
What sits on top
Section titled “What sits on top”The cluster module is the foundation; everything interesting about distributed actor systems comes from the extensions that build on it:
| Extension | What it adds |
|---|---|
| Sharding | One actor per “entity key,” distributed across nodes, with automatic rebalancing on membership changes. |
| Singleton | One actor cluster-wide. Re-spawned elsewhere if the host node leaves. |
| DistributedPubSub | Topic-based fan-out across the cluster. |
| DistributedData | CRDT-based shared state with eventual consistency. |
| Cluster router | Routes messages across cluster-up-members at a well-known path. |
| Receptionist | Service registry — actors register, others look up by key. |
You don’t enable these by default; you reach for them when needed. This page covers the foundation they all share — once you understand membership, gossip, and failure detection, the extensions follow.
Single-node mode
Section titled “Single-node mode”A “cluster” of one node is valid. Calling Cluster.join with
no seeds (or seeds that are unreachable) gives you a singleton
cluster — the local node auto-promotes to leader, becomes up,
and every extension that depends on cluster (sharding, singleton,
pubsub) works as if it were in a larger cluster.
This means cluster code can be developed and tested with a single node; you don’t need a Docker Compose setup to get started. Add more nodes later by giving them seeds pointing at the first.
When to use the cluster
Section titled “When to use the cluster”Three primary motivations:
- Scale-out: more actors than one node’s memory or CPU can handle. Sharding distributes them.
- Fault tolerance: if one node dies, the work moves to another. Singleton and sharding handle the failover.
- Geographic distribution: actors near their users or data, coordinating with the rest of the cluster over the WAN.
For a single-process app, you don’t need the cluster module. For a multi-process app where each process holds independent state, you don’t need it either — just use process boundaries. Reach for it when you genuinely need shared logical state across multiple machines.
Where to next
Section titled “Where to next”- Joining and seeds — the join protocol, seed-node configuration, what happens if all seeds are unreachable.
- Sharding overview — one-actor-per-entity-key with automatic rebalancing.
- Singleton overview — exactly-one cluster-wide.
- Distributed data overview — shared state via CRDTs.
- Failure detector — the elapsed-time heartbeat mechanism.
- Downing strategies — resolving split-brain.
The Cluster API reference covers
the join/leave/subscribe surface.
