Pular para o conteúdo
Português (BR)

Kubernetes API seed provider

Este conteúdo não está disponível em sua língua ainda.

KubernetesApiSeedProvider reads the Endpoints of a named Service from the K8s API and returns the ready pod IPs as seeds. Works without DNS, without SRV, without manual seed-list maintenance — the Service is your contract.

import { Cluster, ClusterOptions } from 'actor-ts/cluster';
import { KubernetesApiSeedProvider, KubernetesApiSeedProviderOptions } from 'actor-ts/discovery';
const kubernetesApiSeedProviderOptions = KubernetesApiSeedProviderOptions.create()
.withNamespace(process.env.K8S_NAMESPACE!)
.withServiceName('actor-ts')
.withSystemName('my-app')
.withPort(2552);
const provider = new KubernetesApiSeedProvider(
kubernetesApiSeedProviderOptions,
);
const seeds = await provider.lookup();
const clusterOptions = ClusterOptions.create()
.withHost('0.0.0.0')
.withAdvertisedHost(process.env.POD_IP!)
.withPort(2552)
.withSeeds(seeds);
await Cluster.join(
system,
clusterOptions,
);

For every ready pod backing the actor-ts Service in the namespace, the provider returns <pod-ip>:2552.

The address pair is the point of the example. A pod does not know its own IP when the process starts, so it binds 0.0.0.0 — but 0.0.0.0 is not an identity, and a fleet of pods that all advertised it would all advertise the same string, each reading the others’ announcements as claims about itself. POD_IP is the address the platform assigned, so that is what goes out in gossip.

POD_IP is not automatic — the pod spec has to export it:

env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP

With that in place withAdvertisedHost is optional: the resolution chain reads POD_IP on its own once the bind host is a wildcard. Naming it is still worth the line — it puts the deployment’s intent in the code rather than in an environment you have to go and check.

type KubernetesApiSeedProviderOptionsType = {
namespace: string;
serviceName: string;
systemName: string;
port: number;
requestTimeoutMs?: number; // ceiling on the API request, default 10 000
fetchEndpoints?: () => Promise<string[]>; // override the in-cluster API
pinnedAddresses?: readonly string[]; // CIDRs the pod IPs must fall inside
log?: (message: string, error?: unknown) => void;
};
FieldWhat
namespaceK8s namespace to query — typically your app’s namespace. A DNS-1123 label.
serviceNameThe Service (or Endpoints) name whose ready pods form the cluster. A DNS-1123 subdomain.
systemNameThe actor-system name stamped on each discovered address.
portThe cluster-transport port on each backing pod.
requestTimeoutMsCeiling on the one request the in-cluster fetcher makes, in ms. Default 10 000. Past it lookup() rejects, so the discovery ladder falls through to its next rung instead of hanging on an API server that accepts and never answers. HOCON: actor-ts.discovery.kubernetes.request-timeout.
fetchEndpointsOverride the Endpoints-fetch function — defaults to the in-cluster API. Also lifts the name rules above.
pinnedAddressesCIDRs the discovered pod IPs must fall inside. Unset means no pinning.
logReports addresses dropped by pinnedAddresses. Default: no-op.

For pods running in-cluster, only the last four fields are optional — the framework reads the Endpoints from the standard API at https://kubernetes.default.svc using the mounted ServiceAccount token. The other four fields are required.

The two names are what the provider puts into its request — GET /api/v1/namespaces/<namespace>/endpoints/<serviceName> — so both segments are percent-encoded, and both are checked against the shape Kubernetes itself requires:

  • namespace — a DNS-1123 label: lowercase alphanumeric or -, starting and ending alphanumeric, at most 63 characters.
  • serviceName — a DNS-1123 subdomain: dot-separated labels of that form, at most 253 characters, because an Endpoints object may carry a dotted name.

Both values come straight out of the pod’s environment when you bootstrap from it (CLUSTER_NAMESPACE / CLUSTER_SERVICE_NAME), so a /, .. or ? reaching them would otherwise address a different API resource with this pod’s ServiceAccount token attached. A name outside the shape is rejected at construction with an OptionsError naming the field, instead of arriving later as a puzzling 404.

Supplying fetchEndpoints lifts the shape rule: your fetcher builds its own request, so there the two fields are plain labels and only have to be non-empty.

On the env-driven ladder — Cluster.bootstrap() with neither seeds nor discovery — a rejected name costs only this rung. The Kubernetes provider is dropped, the rejection is reported through the bootstrap log, and the CLUSTER_SEEDS and DNS rungs still run. That matters because CLUSTER_SERVICE_NAME drives the DNS rung as well, and a DNS hostname is the wider form: an SRV name like _actor-ts._tcp.example.com, a root-anchored FQDN with a trailing dot, and uppercase are all legal there, and none of them is a DNS-1123 subdomain. The pinned form Cluster.bootstrap({ discovery: 'kubernetes' }) keeps failing loudly — pinning one provider is a statement that you want no fallback.

An Endpoints object may name any IP, including one no pod in the cluster owns. pinnedAddresses bounds what a write to that object can redirect the bootstrap towards:

const kubernetesApiSeedProviderOptions = KubernetesApiSeedProviderOptions.create()
.withNamespace(process.env.K8S_NAMESPACE!)
.withServiceName('actor-ts')
.withSystemName('my-app')
.withPort(2552)
.withPinnedAddresses(['10.244.0.0/16'])
.withLog((message) => logger.warn(message));

Entries here are CIDRs only — Endpoints always resolve to IPs, so a host suffix could never match and is rejected at construction, as are an empty list and a malformed CIDR. Addresses outside the list are filtered out and reported through log, one message per dropped address.

This provider is already the better-defended of the two: the default fetcher pins TLS to the ServiceAccount CA, so it does not inherit DNS’s trust problem the way the DNS provider does. RBAC that can write Endpoints is still a much cheaper find than a CA key, which is what this pin costs an attacker.

The pod’s ServiceAccount needs read access to the Service’s Endpoints in the namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: actor-ts-endpoints-reader
namespace: my-app
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: actor-ts-endpoints-reader
namespace: my-app
subjects:
- kind: ServiceAccount
name: actor-ts
roleRef:
kind: Role
name: actor-ts-endpoints-reader
apiGroup: rbac.authorization.k8s.io

Without these, the lookup gets a 403; Cluster.join retries indefinitely.

From the Service’s Endpoints object:

  • Ready pods in subsets[].addresses[] → included as <podIP>:<port>.
  • Not-ready / pending pods (in notReadyAddresses) → skipped (no ready IP yet).
  • This pod itself → may be included or not depending on readiness timing; the cluster handles self-as-seed correctly.

Membership is defined by the Service’s own spec.selector — the provider only needs the Service name:

# One Service fronting every cluster pod:
apiVersion: v1
kind: Service
metadata:
name: actor-ts
spec:
clusterIP: None # headless — Endpoints carry the pod IPs
selector:
app: actor-ts
ports:
- port: 2552

Point the provider at that Service with .withServiceName('actor-ts'). For most deployments, one Service per cluster — every pod the Service selects bootstraps into one cluster.

For role-based asymmetric clusters (workers vs. HTTP gateways), a single Service whose selector (app=actor-ts) covers both roles is enough; the role tag inside the cluster is separate from the Service’s pod selection.

yes

no, first pod

Pod starts → Cluster.join called

KubernetesApiSeedProvider.lookup

GET /api/v1/namespaces/<ns>/
endpoints/<serviceName>

read subsets[].addresses[].ip

return [...podIp:port]

Cluster.join tries each seed

existing cluster?

join successfully

self-bootstrap

If you’re the first pod up, the K8s API returns this pod only. The framework’s self-bootstrap logic handles this: Cluster.join with seeds containing only itself self-promotes to leader.

Subsequent pods see the existing pods and join through them.

Service layoutEffect
One Service, selector app=actor-tsOne cluster across the namespace.
Two Services, selectors env=prod / env=stagingSeparate prod and staging clusters in one namespace.
One Service per named clusterExplicit cluster name; multiple clusters in one namespace.

Stable Service design matters — the cluster’s identity is the Service you point at. Changing which pods the Service selects (or the Service name) splits the cluster.