Cassandra journal

CassandraJournal stores events in a Cassandra cluster. Unlike SQLite (one file per node), Cassandra is shared across cluster nodes — any node can append, read, or query events for any persistenceId.

import { ActorSystem, ActorSystemOptions } from 'actor-ts';
import { CassandraJournal, CassandraJournalOptions } from 'actor-ts/persistence';

const cassandraJournalOptions = CassandraJournalOptions.create()
  .withContactPoints(['cassandra-1:9042', 'cassandra-2:9042'])
  .withKeyspace('my_app_events')
  .withEventsTable('events');
const actorSystemOptions = ActorSystemOptions.create().withPersistence({
  journal: new CassandraJournal(cassandraJournalOptions),
});
const system = ActorSystem.create('my-app', actorSystemOptions);

When to use it

Cassandra is the production choice for multi-node clusters with shared persistence:

• Sharded entities that move between nodes — PersistentActors spawned on different nodes need to read each other’s journals during rebalance.
• Cross-node projections — a projection on node-A needs to see events written on node-B.
• High-throughput single-shard scenarios that exceed SQLite’s per-machine ceiling.

For single-node deployments, SqliteJournal is simpler and cheaper — Cassandra has operational complexity (multi-node cluster, repair, tuning) you don’t need.

Configuration

type CassandraJournalOptions = {
  contactPoints:    string[];      // cluster contact points
  keyspace:         string;        // keyspace (created externally)
  eventsTable?:      string;        // events table name, default 'events'
  metadataTable?:    string;        // max-seq-per-pid table, default 'metadata'
  allIdsTable?:      string;        // persistenceIds() lookup, default 'all_persistence_ids'
  tagIndexTable?:    string;        // tag-index side table, default 'events_by_tag'
  partitionSize?:    number;        // rows per partition, default 500_000
  autoCreateTables?: boolean;       // create tables on first connect, default true
  useTagIndex?:      boolean;       // maintain events_by_tag side table, default false
  consistency?:      number;        // CQL consistency, default LOCAL_QUORUM (6)
  lightweightTransactions?: boolean; // LWT-serialized appends, default true
  serialConsistency?: number;        // Paxos consistency for the LWT claim
  /* ... plus driver-level connection options ... */
};

Field	What
contactPoints	Initial Cassandra contact nodes. Driver discovers the rest.
keyspace	Pre-existing keyspace. The framework creates tables but not the keyspace itself.
eventsTable	Events table name. Default events.
metadataTable	Table tracking the highest sequence number per pid. Default metadata.
allIdsTable	Lookup table backing persistenceIds(). Default all_persistence_ids.
tagIndexTable	Tag-index side table. Default events_by_tag. Only written when useTagIndex is set.
partitionSize	Rows per partition before rolling to a new bucket. Default 500_000.
autoCreateTables	Auto-create the tables on first connect. Default true.
useTagIndex	Maintain the events_by_tag side table for indexed tag queries. Default false.
consistency	CQL consistency level (numeric, from the driver’s types.consistencies) for all reads and writes. Default LOCAL_QUORUM (6).
lightweightTransactions	Serialize concurrent appends with an LWT on the metadata row. Default true. See Concurrent appends.
serialConsistency	Consistency for the LWT’s Paxos phase (numeric, from the driver’s types.consistencies). Unset means the driver’s cluster-wide SERIAL; on a multi-DC keyspace set localSerial (9).

The framework auto-creates three tables on first use — events, metadata, and all_persistence_ids — plus events_by_tag when useTagIndex is enabled. Schemas:

CREATE TABLE events (
  persistence_id  text,
  partition_nr    bigint,
  sequence_nr     bigint,
  timestamp       bigint,
  payload         text,
  tags            set<text>,
  PRIMARY KEY ((persistence_id, partition_nr), sequence_nr)
) WITH CLUSTERING ORDER BY (sequence_nr ASC);

CREATE TABLE metadata (
  persistence_id   text PRIMARY KEY,
  max_sequence_nr  bigint,
  updated_at       bigint
);

CREATE TABLE all_persistence_ids (
  tag             text,
  persistence_id  text,
  PRIMARY KEY (tag, persistence_id)
);

-- Only when useTagIndex is enabled:
CREATE TABLE events_by_tag (
  tag             text,
  timestamp       bigint,
  persistence_id  text,
  sequence_nr     bigint,
  payload         text,
  tags            set<text>,
  PRIMARY KEY ((tag), timestamp, persistence_id, sequence_nr)
) WITH CLUSTERING ORDER BY (timestamp ASC, persistence_id ASC, sequence_nr ASC);

The events table uses a composite partition key (persistence_id, partition_nr): a persistenceId’s events are bucketed into partitions of partitionSize rows (default 500,000), so a long-lived stream spans multiple partitions and recovery reads one partition per bucket. The metadata table tracks the highest sequence number per pid; all_persistence_ids backs persistenceIds(). The optional events_by_tag table is keyed by tag — projection queries hit one partition per tag.

Compaction reaches the side table. events_by_tag is a real table this journal dual-writes, not an index Cassandra maintains, so delete compacts it explicitly: the tag rows of the deleted prefix go first, then the events. That matters more here than it looks — each side-table row carries its own copy of the payload, so a row left behind would keep both answering currentEventsByTag and storing the event body after the event itself was compacted away. Because the side table is partitioned by tag and clustered on timestamp, neither of which is derivable from (persistenceId, toSeq), the delete first reads the compacted prefix’s (sequence_nr, timestamp, tags) back from events to rebuild those keys. Budget one extra SELECT per partition plus one DELETE per (event, tag) pair on a useTagIndex journal; with the index off, delete issues exactly the statements it always did.

keyspace and every table name are interpolated into CQL as identifiers — identifiers cannot be bound — so each is validated against /^[A-Za-z_][A-Za-z0-9_]*$/ before it reaches a statement: on the write path, on the tag-index read path, and inside the CREATE KEYSPACE that autoCreateKeyspace issues. Data-center names in replication.dataCenters are CQL string values rather than identifiers, so they keep the hyphens an Ec2Snitch-derived name carries (us-east-1) and are quote-escaped instead; replication factors must be integers. Everything else — persistenceIds, tags, payloads — travels as bind parameters, never string-concatenated.

Setup

Provision the keyspace with appropriate replication:

CREATE KEYSPACE my_app_events
  WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'datacenter1': 3,
  };

NetworkTopologyStrategy with a replication factor of 3 is typical for production. The framework’s writes go via LOCAL_QUORUM, which needs 2 of 3 replicas for ack.

Consistency model

Cassandra is eventually consistent across replicas — but each pid’s appends are serialized (see Concurrent appends). Practical guarantees:

• A given pid’s events have a strict total order (sequenceNr).
• Replays see events in seq order regardless of which Cassandra replica responds.
• Cross-pid event order in tag queries is timestamp-bound but not strict — events with the same ts may interleave.

For most event-sourced applications, this is fine — within a single entity (pid), order is strict; across entities, partial order via timestamp is acceptable.

Concurrent appends

Every append claims its sequence range with a Cassandra lightweight transaction (LWT) on the metadata row before a single event is written:

-- first append for a pid
INSERT INTO metadata (persistence_id, max_sequence_nr, updated_at)
  VALUES (?, ?, ?) IF NOT EXISTS;

-- every later append
UPDATE metadata SET max_sequence_nr = ?, updated_at = ?
  WHERE persistence_id = ? IF max_sequence_nr = ?;

Only one writer’s claim wins the Paxos round. The others get a JournalConcurrencyError carrying the head the winner left behind — the same contract the relational backends give you.

This is not optional bookkeeping. A CQL INSERT is an upsert: without the claim, two writers that both read head N would both pass the expectedSeq check and both write sequence_nr = N+1, the second silently overwriting the first while both callers are told their event was persisted. The relational journals can’t lose that race — their primary key rejects the loser — so the LWT is what closes the gap for Cassandra.

What it costs

One Paxos round-trip per append, not per event. The claim is a single conditional statement against one small partition; the events themselves still go out in the same unlogged per-partition batch as before. Budget roughly 3–4× the latency of a non-LWT append — Paxos is a prepare/propose/commit sequence across a quorum, versus one round-trip for a normal write. Throughput per pid drops accordingly; throughput across pids is unaffected, because each claim contends only on its own metadata partition.

Claiming on the metadata row is deliberately cheaper than the obvious alternative of INSERT … IF NOT EXISTS on the events: that costs a round-trip per event, and a conditional batch must stay inside a single partition — which the events_by_tag dual-write already rules out.

The crash window

Claiming before writing inverts, rather than removes, the failure window:

• If the event batch fails, the journal issues a compensating release — conditional on the value it claimed, so a writer that has legitimately moved the head on is never rewound — and a retry at the same expectedSeq succeeds.
• If the process dies between the committed claim and the events, the head is left ahead of the stored events: a gap. A replay then reads fewer events than highestSeq reports, and the next append continues past the gap.

The previous ordering had the mirror-image window (events written, head not advanced, orphans overwritten on retry). The gap is the safer of the two to trade for the concurrency guarantee, because it is detectable — compare highestSeq against the highest sequence_nr actually present — whereas a lost write is not.

Opting out

const cassandraJournalOptions = CassandraJournalOptions.create()
  .withContactPoints(['cassandra-1:9042'])
  .withKeyspace('my_app_events')
  .withLightweightTransactions(false);

Only do this if you genuinely guarantee one writer per persistence id and need the round-trip back. Cluster sharding aims for that invariant but does not guarantee it during a rebalance or a split brain — which is exactly when two writers for one pid coexist. With LWT off, a losing concurrent append discards its event and reports success.

Set it uniformly across the fleet. Cassandra only guarantees LWT linearizability when every write to a partition goes through Paxos; a rolling change that leaves some nodes writing metadata conditionally and others writing it plainly breaks the guarantee for the duration of the roll. If you must flip it on a live cluster, drain writers for the affected pids first.

Multi-DC

Cassandra natively supports multi-datacenter replication. Configure replication per DC:

CREATE KEYSPACE my_app_events
  WITH replication = {
    'class': 'NetworkTopologyStrategy',
    'dc1': 3,
    'dc2': 3,
  };

The actor-ts journal doesn’t care — writes go to local DC (via LOCAL_QUORUM), cross-DC replication is async and handled by Cassandra.

Set serialConsistency on a multi-DC keyspace. The append claim is a Paxos round, and the driver’s default serial consistency is cluster-wide SERIAL — a quorum of replicas across every DC. Left alone it drags each append over the inter-DC link, undoing exactly what LOCAL_QUORUM buys you:

import { types } from 'cassandra-driver';

const cassandraJournalOptions = CassandraJournalOptions.create()
  .withContactPoints(['cassandra-1:9042'])
  .withKeyspace('my_app_events')
  .withSerialConsistency(types.consistencies.localSerial);

LOCAL_SERIAL keeps Paxos inside the local DC. The trade is that appends are then linearizable only within a DC — fine when a given pid is written from one DC at a time, which is the usual sharding layout, and unsafe if you route the same pid to two DCs concurrently.

Costs

Approximate write performance (single Cassandra cluster):

• Single-pid append — dominated by the Paxos round that claims the sequence range, so roughly 3–4× a plain quorum write. With lightweightTransactions: false it drops back to sub-millisecond at the journal level, driven by Cassandra’s commit log + memtable — read Concurrent appends before making that trade.
• Cross-pid throughput — scales linearly with cluster size. Each pid contends only on its own metadata partition, so the LWT does not serialize unrelated writers.
• Tag query — bounded by tag partition size. Hot tags (every event tagged ‘audit’) become hot partitions; consider finer-grained tagging or bucketing if you see one tag carrying 100M+ events.

Backup + restore

Cassandra has its own backup strategy — snapshots via nodetool snapshot, incremental backups, plus operational tooling (Medusa, Cassandra Backup tool). The journal doesn’t add anything special; treat it as you would any other Cassandra keyspace.

Pitfalls

Don't run Cassandra single-node in production

contactPoints: ['localhost:9042'];   // ✗ single node

Cassandra’s value is replication. Single-node Cassandra is strictly worse than SQLite — same operational footprint, no durability advantage. Either run a real Cassandra cluster or use SQLite per-node.

Tag partitions can get hot

override tagsFor(event) { return ['audit']; }

Every event tagged ‘audit’ goes to one Cassandra partition. With 10M events/day, that partition gets huge. Consider bucketing (audit-2025-W19) for high-volume tags, or use finer-grained tags.

Recovery cost

Recovery of a pid with 100K events reads 100K rows from one Cassandra partition. Fast (Cassandra is good at this) but not free. Use snapshots aggressively for actors that accumulate.

Schema changes

The framework auto-creates the events tables on first use. Schema changes (adding columns) are a manual ALTER TABLE exercise. Pre-1.0, schema may evolve in framework releases — read the CHANGELOG before upgrading.

Where to next

• Persistence overview — the bigger picture.
• SQLite journal — the single-node alternative.
• In-memory journal — for tests.
• Snapshots — bound the recovery scan.

The CassandraJournal API reference covers the full options.