Kubernetes API seed provider

KubernetesApiSeedProvider reads the Endpoints of a named Service from the K8s API and returns the ready pod IPs as seeds. Works without DNS, without SRV, without manual seed-list maintenance — the Service is your contract.

import { Cluster, ClusterOptions } from 'actor-ts/cluster';
import { KubernetesApiSeedProvider, KubernetesApiSeedProviderOptions } from 'actor-ts/discovery';

const kubernetesApiSeedProviderOptions = KubernetesApiSeedProviderOptions.create()
  .withNamespace(process.env.K8S_NAMESPACE!)
  .withServiceName('actor-ts')
  .withSystemName('my-app')
  .withPort(2552);
const provider = new KubernetesApiSeedProvider(
  kubernetesApiSeedProviderOptions,
);

const seeds = await provider.lookup();
const clusterOptions = ClusterOptions.create()
  .withHost('0.0.0.0')
  .withAdvertisedHost(process.env.POD_IP!)
  .withPort(2552)
  .withSeeds(seeds);
await Cluster.join(
  system,
  clusterOptions,
);

For every ready pod backing the actor-ts Service in the namespace, the provider returns <pod-ip>:2552.

The address pair is the point of the example. A pod does not know its own IP when the process starts, so it binds 0.0.0.0 — but 0.0.0.0 is not an identity, and a fleet of pods that all advertised it would all advertise the same string, each reading the others’ announcements as claims about itself. POD_IP is the address the platform assigned, so that is what goes out in gossip.

POD_IP is not automatic — the pod spec has to export it:

env:
  - name: POD_IP
    valueFrom:
      fieldRef:
        fieldPath: status.podIP

With that in place withAdvertisedHost is optional: the resolution chain reads POD_IP on its own once the bind host is a wildcard. Naming it is still worth the line — it puts the deployment’s intent in the code rather than in an environment you have to go and check.

Configuration

type KubernetesApiSeedProviderOptionsType = {
  namespace:       string;
  serviceName:     string;
  systemName:      string;
  port:            number;
  fetchEndpoints?: () => Promise<string[]>;  // override the in-cluster API
  pinnedAddresses?: readonly string[];       // CIDRs the pod IPs must fall inside
  log?: (message: string, error?: unknown) => void;
};

Field	What
namespace	K8s namespace to query — typically your app’s namespace. A DNS-1123 label.
serviceName	The Service (or Endpoints) name whose ready pods form the cluster. A DNS-1123 subdomain.
systemName	The actor-system name stamped on each discovered address.
port	The cluster-transport port on each backing pod.
fetchEndpoints	Override the Endpoints-fetch function — defaults to the in-cluster API. Also lifts the name rules above.
pinnedAddresses	CIDRs the discovered pod IPs must fall inside. Unset means no pinning.
log	Reports addresses dropped by pinnedAddresses. Default: no-op.

For pods running in-cluster, only the last three fields are optional — the framework reads the Endpoints from the standard API at https://kubernetes.default.svc using the mounted ServiceAccount token. The other four fields are required.

Names go into the API path

The two names are what the provider puts into its request — GET /api/v1/namespaces/<namespace>/endpoints/<serviceName> — so both segments are percent-encoded, and both are checked against the shape Kubernetes itself requires:

• namespace — a DNS-1123 label: lowercase alphanumeric or -, starting and ending alphanumeric, at most 63 characters.
• serviceName — a DNS-1123 subdomain: dot-separated labels of that form, at most 253 characters, because an Endpoints object may carry a dotted name.

Both values come straight out of the pod’s environment when you bootstrap from it (CLUSTER_NAMESPACE / CLUSTER_SERVICE_NAME), so a /, .. or ? reaching them would otherwise address a different API resource with this pod’s ServiceAccount token attached. A name outside the shape is rejected at construction with an OptionsError naming the field, instead of arriving later as a puzzling 404.

Supplying fetchEndpoints lifts the shape rule: your fetcher builds its own request, so there the two fields are plain labels and only have to be non-empty.

On the env-driven ladder — Cluster.bootstrap() with neither seeds nor discovery — a rejected name costs only this rung. The Kubernetes provider is dropped, the rejection is reported through the bootstrap log, and the CLUSTER_SEEDS and DNS rungs still run. That matters because CLUSTER_SERVICE_NAME drives the DNS rung as well, and a DNS hostname is the wider form: an SRV name like _actor-ts._tcp.example.com, a root-anchored FQDN with a trailing dot, and uppercase are all legal there, and none of them is a DNS-1123 subdomain. The pinned form Cluster.bootstrap({ discovery: 'kubernetes' }) keeps failing loudly — pinning one provider is a statement that you want no fallback.

Pinning the pod CIDR

An Endpoints object may name any IP, including one no pod in the cluster owns. pinnedAddresses bounds what a write to that object can redirect the bootstrap towards:

const kubernetesApiSeedProviderOptions = KubernetesApiSeedProviderOptions.create()
  .withNamespace(process.env.K8S_NAMESPACE!)
  .withServiceName('actor-ts')
  .withSystemName('my-app')
  .withPort(2552)
  .withPinnedAddresses(['10.244.0.0/16'])
  .withLog((message) => logger.warn(message));

Entries here are CIDRs only — Endpoints always resolve to IPs, so a host suffix could never match and is rejected at construction, as are an empty list and a malformed CIDR. Addresses outside the list are filtered out and reported through log, one message per dropped address.

This provider is already the better-defended of the two: the default fetcher pins TLS to the ServiceAccount CA, so it does not inherit DNS’s trust problem the way the DNS provider does. RBAC that can write Endpoints is still a much cheaper find than a CA key, which is what this pin costs an attacker.

RBAC

The pod’s ServiceAccount needs read access to the Service’s Endpoints in the namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: actor-ts-endpoints-reader
  namespace: my-app
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: actor-ts-endpoints-reader
  namespace: my-app
subjects:
  - kind: ServiceAccount
    name: actor-ts
roleRef:
  kind: Role
  name: actor-ts-endpoints-reader
  apiGroup: rbac.authorization.k8s.io

Without these, the lookup gets a 403; Cluster.join retries indefinitely.

What it returns

From the Service’s Endpoints object:

• Ready pods in subsets[].addresses[] → included as <podIP>:<port>.
• Not-ready / pending pods (in notReadyAddresses) → skipped (no ready IP yet).
• This pod itself → may be included or not depending on readiness timing; the cluster handles self-as-seed correctly.

Cluster-wide vs replica-set

Membership is defined by the Service’s own spec.selector — the provider only needs the Service name:

# One Service fronting every cluster pod:
apiVersion: v1
kind: Service
metadata:
  name: actor-ts
spec:
  clusterIP: None        # headless — Endpoints carry the pod IPs
  selector:
    app: actor-ts
  ports:
    - port: 2552

Point the provider at that Service with .withServiceName('actor-ts'). For most deployments, one Service per cluster — every pod the Service selects bootstraps into one cluster.

For role-based asymmetric clusters (workers vs. HTTP gateways), a single Service whose selector (app=actor-ts) covers both roles is enough; the role tag inside the cluster is separate from the Service’s pod selection.

What happens at startup

If you’re the first pod up, the K8s API returns this pod only. The framework’s self-bootstrap logic handles this: Cluster.join with seeds containing only itself self-promotes to leader.

Subsequent pods see the existing pods and join through them.

One Service per cluster

Service layout	Effect
One Service, selector app=actor-ts	One cluster across the namespace.
Two Services, selectors env=prod / env=staging	Separate prod and staging clusters in one namespace.
One Service per named cluster	Explicit cluster name; multiple clusters in one namespace.

Stable Service design matters — the cluster’s identity is the Service you point at. Changing which pods the Service selects (or the Service name) splits the cluster.

When NOT

Pods without IPs

Not-ready pods aren’t in the Service’s Endpoints addresses — the provider skips them. Until at least one pod is Ready, the lookup returns an empty list. This is rarely a problem (first pod self-bootstraps), but be aware during fast scale-ups.

Cross-namespace lookup

namespace: 'other-namespace';   // ✗ unusual; check RBAC

The default RBAC binding above scopes to one namespace. For cross-namespace lookups, use ClusterRole + ClusterRoleBinding, or a Role per source namespace.

K8s API rate limits

// 1000 pods restarting simultaneously → 1000 lookups → API throttling

The K8s API has per-client rate limits. For very large deployments, stagger startup (StatefulSet podManagementPolicy: OrderedReady) to avoid thundering-herd lookups.

Where to next

• Discovery overview — the bigger picture.
• Kubernetes deployment — the full K8s recipe.
• Config seed provider — fallback for non-K8s.
• Aggregate seed provider — combine K8s with a fallback.
• Joining and seeds — how seeds are consumed.