Sharding with lease
Ce contenu n’est pas encore disponible dans votre langue.
The sharding coordinator runs as a cluster singleton — only the leader’s coordinator is active. With a downing strategy + healthy network, that’s enough.
During a network partition + buggy downing config, both halves might briefly think they’re leader → two coordinators → conflicting shard allocations → entities possibly spawned on both sides.
The single-writer lease prevents this:
import { ClusterSharding, StartShardingOptions } from 'actor-ts/cluster';import { KubernetesLease, KubernetesLeaseOptions } from 'actor-ts/coordination';
const kubernetesLeaseOptions = KubernetesLeaseOptions.create() .withName('order-sharding-coordinator') .withOwner(process.env.POD_NAME!) .withTtlMs(30_000) .withNamespace(process.env.K8S_NAMESPACE!);const startShardingOptions = StartShardingOptions.create<Command>() .withTypeName('order') .withEntityActor(OrderEntity) .withExtractEntityId((message) => message.id) .withLease(new KubernetesLease( kubernetesLeaseOptions, ));const sharding = cluster.sharding.start( startShardingOptions,);Now even if two nodes both think they’re leader, only one holds the lease. Only the lease-holder’s coordinator processes shard allocations.
How it works
Section titled “How it works”The lease backend (K8s API server, etcd) provides the atomic exactly-one-holder guarantee — it’s the source of truth beyond gossip.
Configuration
Section titled “Configuration”The lease fields, beside the base sharding settings:
type StartShardingOptionsType<TMessage> = { lease?: Lease; useLease?: boolean; acquireRetryIntervalMs?: number;};| Field | Purpose |
|---|---|
lease | The Lease instance — typically KubernetesLease. Wins over useLease. |
useLease | Build the coordinator’s lease from actor-ts.coordination.lease instead. Default false; also settable node-wide as actor-ts.sharding.use-lease. |
acquireRetryIntervalMs | Retry cadence on failed acquire, default 5000 ms. Also settable node-wide as actor-ts.sharding.acquire-retry-interval. |
From the config file
Section titled “From the config file”actor-ts.coordination.lease { backend = kubernetes ttl = 30s}actor-ts.sharding { use-lease = on acquire-retry-interval = 2s}use-lease = on gives every sharded type this node starts a coordinator lease
the framework builds from the coordination block — named
<system>-sharding-<type> and owned by this node, its cluster address and
incarnation — so the file picks the backend and nothing about identity. Name
and owner are the values a file every node shares cannot carry: one name for
the block would put every type on one record, and one owner on every node
would let each believe it held the lease. With backend = none the switch is
refused when the type starts, rather than advertising split-brain protection
that is not there. A withLease(…) in code still wins, and
withUseLease(false) opts one type out. See
leases the framework builds.
The retry interval is worth tuning because it is the recovery latency of a
partition that heals: until an acquire succeeds the coordinator issues no
AllocateShard, and everything routed for the type waits in the regions’
buffers meanwhile.
Same Lease abstraction as
singleton with lease —
see Coordination for the
interface.
What’s protected
Section titled “What’s protected”The lease gates coordinator state writes:
- Shard allocations — assigning shards to regions.
- Rebalance decisions — moving shards between regions.
- Handoff coordination — orchestrating shard handoffs.
The lease doesn’t gate:
- Per-region entity hosting (regions are tied to actual cluster members, not the coordinator).
- Entity messaging (messages route via the coordinator’s last-known allocation; the lease isn’t in the message path).
So the worst-case during a partition is slightly stale shard assignments — entities continue running, just no new allocations until lease ownership stabilizes.
When to enable
Section titled “When to enable”Three good fits:
- Production multi-region clusters where partitions are plausible.
- Financial / inventory entities where dual-allocation would cause real damage.
- Compliance requiring “no possibility of split-brain in any single-tenant production system.”
For typical single-region K8s deployments with a downing strategy, the lease is paranoid-safe — adds operational complexity for protection against rare edge cases.
Operational considerations
Section titled “Operational considerations”Lease backend availability
Section titled “Lease backend availability”K8s API server outage → no replica can acquire → no shard allocationsThe lease backend becomes a SPOF. For most setups, K8s API availability is much higher than the cluster itself — but plan for the rare case.
Failover latency
Section titled “Failover latency”Old coordinator loses lease (TTL expiry: 30s) → new coordinator acquires (typically sub-second after TTL) → new coordinator rebuilds state from gossip + journalThe failover window is the lease TTL — ~30 s typical. During that window:
- No new shard allocations happen.
- Existing entities continue receiving messages.
- New entity IDs that need allocation queue up; processed after failover.
Acceptable for most workloads.
Combining with regular downing
Section titled “Combining with regular downing”{ downingProvider: new KeepMajority(), // + the sharding lease lease: ...,}Both layers active. Downing handles normal cluster convergence; the lease guarantees the coordinator-uniqueness invariant.
Reading the protection level
Section titled “Reading the protection level”| Setup | Coordinator-uniqueness guarantee |
|---|---|
| No downing, no lease | Best-effort. Partitions cause dual coordinators. |
| Downing strategy only | Strong on stable networks. |
| Downing + lease | Paranoid-safe. Both invariants enforced. |
For singleton + sharding production setups in critical-data scenarios, both is the recommended pattern.
Where to next
Section titled “Where to next”- Sharding overview — the foundation.
- Singleton with lease — the same pattern for singletons.
- Coordination overview — the lease abstraction.
- KubernetesLease — the production backend.
- Downing strategies — the complementary partition resolver.
