Kubernetes deployment
此内容尚不支持你的语言。
Kubernetes is the most common deployment target. The framework plays well with K8s once you get a few things right: stable identity (for stateful actors), seed discovery (so nodes find each other), and a clean shutdown path (so rolling updates don’t drop traffic).
This page is a working recipe — copy, adapt, deploy.
The full manifest
Section titled “The full manifest”apiVersion: v1kind: ServiceAccountmetadata: name: actor-ts---apiVersion: rbac.authorization.k8s.io/v1kind: Rolemetadata: name: actor-ts-pod-readerrules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "list"]---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata: name: actor-tssubjects: - kind: ServiceAccount name: actor-tsroleRef: kind: Role name: actor-ts-pod-reader apiGroup: rbac.authorization.k8s.io---apiVersion: v1kind: Servicemetadata: name: actor-ts-clusterspec: clusterIP: None # headless — DNS returns pod IPs selector: app: actor-ts ports: - name: cluster port: 2552 targetPort: 2552---apiVersion: v1kind: Servicemetadata: name: actor-tsspec: selector: app: actor-ts ports: - name: http port: 80 targetPort: 8080 - name: management port: 8558 targetPort: 8558---apiVersion: apps/v1kind: StatefulSetmetadata: name: actor-tsspec: serviceName: actor-ts-cluster replicas: 3 selector: matchLabels: app: actor-ts template: metadata: labels: app: actor-ts spec: serviceAccountName: actor-ts terminationGracePeriodSeconds: 30 containers: - name: app image: ghcr.io/your-org/your-app:1.2.3 ports: - name: cluster containerPort: 2552 - name: http containerPort: 8080 - name: management containerPort: 8558 env: - name: ACTOR_TS_HOSTNAME valueFrom: fieldRef: fieldPath: status.podIP - name: ACTOR_TS_PORT value: "2552" - name: K8S_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: K8S_LABEL_SELECTOR value: "app=actor-ts" - name: DB_PASSWORD valueFrom: secretKeyRef: name: actor-ts-secrets key: db-password readinessProbe: httpGet: path: /ready port: management initialDelaySeconds: 5 periodSeconds: 5 livenessProbe: httpGet: path: /health port: management initialDelaySeconds: 15 periodSeconds: 10 lifecycle: preStop: exec: # Drain LB before SIGTERM hits the app command: ["/bin/sh", "-c", "sleep 10"]What each piece does
Section titled “What each piece does”ServiceAccount + RBAC
Section titled “ServiceAccount + RBAC”ServiceAccount: actor-tsRole: actor-ts-pod-reader # get + list podsRoleBinding: binds themThe K8s API seed provider needs to list pods matching a label selector to discover peers. Without this RBAC grant, the seed provider gets 403 and the cluster never forms.
Headless Service for cluster gossip
Section titled “Headless Service for cluster gossip”clusterIP: NoneA headless service returns the pod IPs directly via DNS (no ClusterIP virtual address). Useful when nodes need stable, direct peer identities — the failure detector’s heartbeats target specific pod IPs, not a load-balanced abstraction.
Regular Service for traffic
Section titled “Regular Service for traffic”clusterIP: <default>A normal Service for HTTP and the management endpoint — these benefit from load-balancing. Cluster traffic goes through the headless service, app traffic through this one.
StatefulSet vs Deployment
Section titled “StatefulSet vs Deployment”| Use | When |
|---|---|
| StatefulSet | Stable pod names (actor-ts-0, actor-ts-1, …). Useful when you want predictable identity for entity placement, or when persistent volumes are mounted per-pod. |
| Deployment | Pod names are random. Fine if your app is stateless (no per-pod identity required) and persistence is external (Cassandra journal, shared S3 snapshot store). |
For sharded actors with remember-entities = true on persistent
volumes per pod, StatefulSet is the right choice. For
externally-persisted state (cluster talking to a shared
Postgres/Cassandra), Deployment is fine and simpler.
terminationGracePeriodSeconds
Section titled “terminationGracePeriodSeconds”terminationGracePeriodSeconds: 30K8s sends SIGTERM, then waits this long before SIGKILL. Sized based on:
- HTTP drain — typically 5-10 s.
- Cluster leave gossip — 5-15 s for convergence.
- Journal flush — depends on the journal.
30 s is a reasonable default. Bump it if your cluster is large or the failure-detector window is long.
preStop sleep
Section titled “preStop sleep”preStop: exec: command: ["/bin/sh", "-c", "sleep 10"]Critical for clean rolling updates. The flow:
- K8s marks the pod terminating and starts the
preStophook in parallel with the load-balancer-deregistration. sleep 10— gives the load balancer time to stop sending new traffic to this pod.- After the sleep, K8s sends SIGTERM.
- The app’s coordinated-shutdown hooks drain in-flight requests, leave the cluster, etc.
Without the sleep, SIGTERM races with LB deregistration — in-flight requests can see “draining” responses.
Readiness + liveness probes
Section titled “Readiness + liveness probes”readinessProbe: /readylivenessProbe: /healthThe framework’s management routes expose these endpoints.
/ready— “should a load balancer send this pod traffic?” Already gated on the framework’s own checks:cluster-membership(this node isup) andcluster-transport(it is not cut off from every peer), plusstartupwhile startup tasks run. To keep a node out of rotation until its shard regions are registered, name the types inactor-ts.sharding.healthcheck.names(see the built-in checks). Add your per-app checks — database reachable, dependencies warm — withhealthChecksOf(system).addReadiness. In-process,Cluster.bootstrapgates the same way:actor-ts.cluster.bootstrap.minimum-members(orawaitReady: { minimumMembers }in code) keeps a resolved bootstrap from meaning anything less than “the cluster reached its expected size” — set it to the replica count, exactly likerequired-contact-points. On a system with startup tasks it is gated onstartuptoo, which stays false until the startup graph has run every phase — a pod whose startup tasks have not finished takes no traffic, even though its port is already bound./health— “would restarting this pod help?” Failing means K8s restarts it, so it depends on nothing outside the process — the framework’s only liveness check isactor-system. Never put a database or a downstream service here: a shared outage would restart the whole fleet, and the restarts would not fix it.- A failed startup ends the process rather than leaving it
unready. The system shuts itself down through the coordinated
shutdown,
runUntilTerminated()rejects with the startup’s error and the container exits non-zero — so Kubernetes restarts it, and a rollout whose startup cannot succeed stalls at its first pod instead of serving until the first request that needed what startup could not do. - The persistence stores open during startup, so a bad connection
string — a wrong host, a wrong password, a driver the image lacks —
fails the pod at boot, not at its first persisted message
(opening the stores at
startup) —
except behind the object-storage snapshot and durable-state stores,
which open nothing ahead of time, so an unreachable bucket fails
their first operation instead.
The other side of it: a database that becomes reachable only after
the app starts sends the pod into
CrashLoopBackOffuntil it is, each attempt failing when the store is refused, or when thepersistence-startphase’s 30 s budget runs out. If the two are meant to start independently, bring the database up first, or set the persistence behaviour’sautoStartto false and let the first persisted message find it.
App-side wiring
Section titled “App-side wiring”import { ActorSystem } from 'actor-ts';import { Cluster, ClusterOptions } from 'actor-ts/cluster';import { KubernetesApiSeedProvider, KubernetesApiSeedProviderOptions } from 'actor-ts/discovery';import { managementRoutes } from 'actor-ts/management';
const system = ActorSystem.create('my-app');
// 1. Cluster join with K8s API seed discoveryconst kubernetesApiSeedProviderOptions = KubernetesApiSeedProviderOptions.create() .withNamespace(process.env.K8S_NAMESPACE!) .withServiceName(process.env.K8S_SERVICE_NAME!) .withSystemName(system.name) .withPort(2552);const seeds = await new KubernetesApiSeedProvider(kubernetesApiSeedProviderOptions).lookup();
const clusterOptions = ClusterOptions.create() .withHost(process.env.ACTOR_TS_HOSTNAME!) .withPort(parseInt(process.env.ACTOR_TS_PORT!)) .withSeeds(seeds.map(a => a.toString())) .withRoles(['compute']);const cluster = await Cluster.join(system, clusterOptions);
// 2. Management endpointsconst mgmtRoutes = managementRoutes(system, cluster);await system.http(8558).bind(mgmtRoutes);
// 3. App HTTP serverconst http = system.extension(HttpExtensionId);await http.newServerAt('0.0.0.0', 8080).bind(routes);
// 4. Run until the kubelet says stop, then shut down in orderawait system.runUntilTerminated();Four pieces in order:
- Cluster join with seed discovery — the K8s API seed
provider queries pods matching
app=actor-tsand uses their IPs as seeds. On the first pod, the seed list is just itself (the auto-promote-to-leader path). - Management endpoints —
/health+/readyfor K8s probes,/cluster/membersfor debugging. - App HTTP — your routes, separate port from management.
- Run until terminated — installs the SIGTERM/SIGINT handlers,
blocks, then runs the pipeline and resolves when the system is
down. There is no separate teardown step to write: both
bind()calls registered their unbind in theservice-unbindphase andCluster.joinregistered the leave incluster-leave, so the pod drops out of the Service endpoints and out of the cluster before the actors stop.
See Discovery — Kubernetes API for the seed provider’s full options.
Secrets as a mounted file
Section titled “Secrets as a mounted file”The manifest above hands DB_PASSWORD to the process as an environment
variable. The secrets file keeps
credentials out of the environment altogether: a Secret mounted as a
file, named by ACTOR_TS_SECRETS_CONF, with the
profile chosen by
ACTOR_TS_PROFILE:
apiVersion: v1kind: Secretmetadata: name: actor-ts-secrets-conftype: OpaquestringData: secrets.conf: | my-app.db.password = "s3cr3t"---# The StatefulSet above, under spec.template.spec:containers: - name: app env: - name: ACTOR_TS_PROFILE value: "production" - name: ACTOR_TS_SECRETS_CONF value: /etc/actor-ts/secrets.conf volumeMounts: - name: secrets-conf mountPath: /etc/actor-ts readOnly: truevolumes: - name: secrets-conf secret: secretName: actor-ts-secrets-conf- Withheld whatever it is called. Every value the file sets is
replaced with
<redacted>in the config dump and the DevTools panel, in worker threads too; a variable substituted intoapplication.confis withheld only when its key names a secret. - One image, every environment.
ACTOR_TS_PROFILElays theprofiles.productionblock ofapplication.conf— and of the secrets file — over its root, and an unknown profile stops the start. - Read once, at startup. Kubernetes refreshes a mounted
Secretin place, but the configuration is not re-read; roll the pods to pick up a change. - A missing file is not an error. Without the mount the secrets
layer is just empty, and the dump’s header says the file was not found.
Give a credential no default in
application.conf, so its absence fails where it is read rather than running on a placeholder.
The first deployment, and every one after
Section titled “The first deployment, and every one after”If the pods start through Cluster.bootstrap rather than the hand-wired
join above, the very first rollout and every rollout after it want one
setting apart. On the first, some pod has to found the cluster — which is
what actor-ts.cluster.bootstrap.new-cluster-enabled ships as. Once the
cluster exists, turn it off. Reading it from a variable keeps that a
one-line change to the manifest rather than a new image:
actor-ts.cluster.bootstrap.new-cluster-enabled = ${?ACTOR_TS_NEW_CLUSTER_ENABLED}# The StatefulSet above, under spec.template.spec.containers[0].env —# added after the first rollout has formed the cluster:- name: ACTOR_TS_NEW_CLUSTER_ENABLED value: "off"From then on a pod that cannot reach its peers at start-up — a NetworkPolicy change, an API-server hiccup, a DNS answer that lags pod readiness — fails its readiness wait and is restarted, instead of coming up as the founder of a second cluster beside the running one. Every start logs a warning while the key is still on, so a deployment that never flipped it says so in its own logs. See Cluster bootstrap.
Rolling updates
Section titled “Rolling updates”kubectl rollout restart statefulset/actor-tsFor each pod, in order (StatefulSet) or arbitrary (Deployment):
- K8s marks the pod terminating + starts
preStop. - 10-second LB drain.
- SIGTERM lands.
- Coordinated-shutdown runs:
- Stop accepting new HTTP requests (
service-unbind, wired bybind()). - Drain in-flight requests (
service-requests-done, yours). - Close broker connections (
service-stop, wired by each broker actor). - Issue
cluster.leave()(cluster-leave, wired byCluster.join). - Terminate the actor system (
actor-system-terminate).
- Stop accepting new HTTP requests (
runUntilTerminated()resolves, its signal handlers come off, and the process exits cleanly.- K8s starts a new pod from the new image.
- New pod joins the cluster via the seed provider.
For sharded entities, rebalancing happens automatically — the leaving node’s shards are reallocated; new entities re-spawn on the new pod from the journal.
Where to next
Section titled “Where to next”- Operations overview — the bigger picture across all production concerns.
- Discovery — Kubernetes API — the seed provider’s options.
- Coordinated shutdown — the phased-shutdown DSL.
- Management endpoints — the /health + /ready endpoints K8s probes hit.
- Cluster security — TLS + auth for the cluster transport.
- Rolling migration — schema-breaking changes during a rollout.
