コンテンツにスキップ
日本語

Compression

このコンテンツはまだ日本語訳がありません。

Object storage supports at-rest compression — bodies are compressed before put, decompressed on get. Three modes; pick the CPU-vs-size trade that fits the payload.

import {
ObjectStorageDurableStateStore,
ObjectStorageDurableStateStoreOptions,
S3ObjectStorageBackend,
S3ObjectStorageOptions,
} from 'actor-ts/persistence';
const objectStorageDurableStateStoreOptions = ObjectStorageDurableStateStoreOptions.create()
.withBackend(new S3ObjectStorageBackend(S3ObjectStorageOptions.create() /* .withRegion(...).withBucket(...) */))
.withCompression({
algorithm: 'zstd',
level: 6, // optional; zstd 1–22 (default 3)
});
const store = new ObjectStorageDurableStateStore(objectStorageDurableStateStoreOptions);

Now every persisted state is zstd-compressed before upload. Reads transparently decompress.

AlgorithmCompression ratioCPU costWhen
zstdBestLow–moderate (decode very fast)Large state blobs — best ratio at competitive speed.
gzipGood (typical 50-70 %)ModerateUniversal default — node:zlib, works on every runtime.
noneNoneZeroAlready-compressed or tiny payloads (see caveat below).

The store default is gzip — it needs no native runtime support and no extra dependency. For text-heavy state (JSON), both gzip and zstd reach ~70 % reduction; zstd gets there faster and decompresses quicker. For already-compressed data (encrypted bytes, image data), compression is nearly free of value and wastes CPU — use none.

There is no brotli or deflate mode — only none, gzip, and zstd.

type CompressionAlgo = 'none' | 'gzip' | 'zstd';
type CompressionConfig = {
algorithm: CompressionAlgo;
level?: number; // algorithm-specific; clamped; undefined = default
};

Levels are algorithm-specific. Out-of-range values are clamped; omitting level uses the implementation default:

  • gzip: 0 (no compression) – 9 (best). Default 6.
  • zstd: 1 (fastest) – 22 (best). Default 3.
  • none: ignored.

For most workloads the defaults are fine — bump only when storage cost dominates.

zstd support differs by direction:

  • Compress (write) — native only: Bun (Bun.zstdCompressSync) or Node (zlib.zstdCompressSync). There is no pure-JS fallback for writing; the optional fzstd peer dependency is decompress-only. Selecting zstd on a runtime without native support fails fast — eagerly at plugin registration, not cryptically on first persist.
  • Decompress (read) — node:zlib first (it is the only zstd implementation that takes an allocation-time output bound — see Decompression cap), then Bun’s global, then the optional fzstd peer-dep, so a runtime without native zstd can still read zstd bodies written elsewhere once fzstd is installed.

Both directions pick an implementation by calling it, not by checking that the function exists — the two differ on Deno, whose node:zlib exports zstd entry points with no binding behind them. So a runtime without working zstd gets the framework’s own message naming the missing capability, and the fzstd fallback below it stays reachable.

gzip uses node:zlib and works everywhere (Bun, Node, Deno) with no extra dependency.

On put:
serialize value → JSON bytes → compress(bytes) → [encrypt] →
frame with ATS1 manifest → backend.put(framed)
On get:
backend.get → framed bytes → read ATS1 manifest → [decrypt] →
decompress per the manifest's algorithm flag → JSON → deserialize

Every body carries a small ATS1 manifest header; a 2-bit field in its flags byte records the compression algorithm (none / gzip / zstd). Decompression is driven entirely by that flag — not by any HTTP header. (On S3 the framework also sets Content-Encoding as a courtesy marker, but it is never the source of truth for decoding.)

Because each body self-describes its algorithm, mixing none / gzip / zstd bodies in the same bucket works — every object decodes per its own manifest.

Decompression on read is bounded to 512 MiB of output by default, so a tampered or hostile compressed body (a few KB that inflates to many GB) can’t exhaust memory during recovery (security audit #3).

gzip and zstd hand the bound to the decoder via zlib’s maxOutputLength, so an over-cap body is refused before its output is allocated — the bomb never gets the memory it was built to claim. Every path additionally asserts the decoded size, which is what bounds the two that cannot take the option (Bun’s Bun.zstdDecompressSync, reached only on a Bun without zstd in node:zlib, and the fzstd fallback). There the cap is still enforced, but only after the output exists — so on a runtime that falls back to either, treat the bound as a correctness check rather than as bomb protection.

Either way the read fails with an error instead of OOM-ing the process, and the error names the bound (maxOutputBytes=…) so it is clear which setting to reach for. Snapshots / durable state are far smaller than this in practice; if you legitimately store larger bodies, raise the bound with withMaxDecompressedBytes on the store (or plugin) options:

const objectStorageDurableStateStoreOptions = ObjectStorageDurableStateStoreOptions.create()
.withBackend(backend)
.withMaxDecompressedBytes(2 * 1024 * 1024 * 1024); // 2 GiB; Infinity opts out

withMaxDecompressedBytes is available on the snapshot-store, durable-state-store, and plugin options; the value is validated at construction (a non-positive / non-integer cap throws OptionsError).

The level is an encoder-only setting. It is not stored on the wire (the manifest records the algorithm, not the level), and the decompressor never needs it — gzip and zstd frames are self-describing.

So changing the configured level requires no migration: bodies written at the old level keep decoding, new bodies use the new level, and the two mix freely. You can also raise or lower the level per-actor at any time without touching existing data.

serialize → JSON bytes → compress → [encrypt] → store

Compression runs after serialization and before encryption — compressing ciphertext is pointless (it’s high-entropy), and the fixed order also avoids CRIME-style side channels. For maximum size reduction on large text-heavy state, zstd at a higher level is usually the best single knob to turn.

ScenarioAlgorithm
Large text-heavy state (big JSON, long descriptions)zstd
Mixed text + numberszstd or gzip
Maximum portability, no native zstd guaranteedgzip
Already-encrypted / already-compressed bytesnone (no benefit)
Storage-cost-bounded, write-once-read-manyzstd (level 15-19)
CPU-bounded write pathgzip level 1 or zstd level 1

Single thread, ~100 KB blob — order-of-magnitude only; real numbers depend on payload and runtime:

  • gzip level 6 — ~1-3 ms encode, ~0.5 ms decode.
  • gzip level 1 — ~0.5 ms encode, fast decode.
  • zstd level 3 (default) — encode comparable to gzip, better ratio, very fast decode.
  • zstd level 19 — noticeably slower encode, decode still fast.
  • zstd levels 20-22 (ultra) — much slower encode for marginal gains; reserve for write-once-read-many bulk data.

zstd’s asymmetry (cheap decode at every level) makes it a strong default for read-heavy state. For typical state-store workloads (small objects, modest write rate) the CPU cost is invisible.

class SmallStateActor extends DurableStateActor<...> {
protected compression() { return { algorithm: 'none' as const }; }
}
class LogStateActor extends DurableStateActor<...> {
protected compression() { return { algorithm: 'zstd' as const, level: 19 }; }
}

Override the store-level default per actor. Useful when:

  • Most actors have small state (no compression needed).
  • A few have large text-heavy state (high compression).

See Per-actor policies.