Compression
Ce contenu n’est pas encore disponible dans votre langue.
Object storage supports at-rest compression — bodies are compressed before put, decompressed on get. Three modes; pick the CPU-vs-size trade that fits the payload.
import { ObjectStorageDurableStateStore, ObjectStorageDurableStateStoreOptions, S3ObjectStorageBackend, S3ObjectStorageOptions,} from 'actor-ts/persistence';
const objectStorageDurableStateStoreOptions = ObjectStorageDurableStateStoreOptions.create() .withBackend(new S3ObjectStorageBackend(S3ObjectStorageOptions.create() /* .withRegion(...).withBucket(...) */)) .withCompression({ algorithm: 'zstd', level: 6, // optional; zstd 1–22 (default 3) });const store = new ObjectStorageDurableStateStore(objectStorageDurableStateStoreOptions);Now every persisted state is zstd-compressed before upload. Reads transparently decompress.
Algorithms
Section titled “Algorithms”| Algorithm | Compression ratio | CPU cost | When |
|---|---|---|---|
zstd | Best | Low–moderate (decode very fast) | Large state blobs — best ratio at competitive speed. |
gzip | Good (typical 50-70 %) | Moderate | Universal default — node:zlib, works on every runtime. |
none | None | Zero | Already-compressed or tiny payloads (see caveat below). |
The store default is gzip — it needs no native runtime
support and no extra dependency. For text-heavy state
(JSON), both gzip and zstd reach ~70 % reduction; zstd gets there
faster and decompresses quicker. For already-compressed data
(encrypted bytes, image data), compression is nearly free of
value and wastes CPU — use none.
There is no
brotliordeflatemode — onlynone,gzip, andzstd.
Configuration
Section titled “Configuration”type CompressionAlgo = 'none' | 'gzip' | 'zstd';
type CompressionConfig = { algorithm: CompressionAlgo; level?: number; // algorithm-specific; clamped; undefined = default};Levels are algorithm-specific. Out-of-range values are
clamped; omitting level uses the implementation default:
- gzip: 0 (no compression) – 9 (best). Default 6.
- zstd: 1 (fastest) – 22 (best). Default 3.
- none: ignored.
For most workloads the defaults are fine — bump only when storage cost dominates.
Runtime support
Section titled “Runtime support”zstd support differs by direction:
- Compress (write) — native only: Bun (
Bun.zstdCompress) or Node (zlib.zstdCompress). There is no pure-JS fallback for writing; the optionalfzstdpeer dependency is decompress-only. Selectingzstdon a runtime without native support fails fast — eagerly at plugin registration, not cryptically on first persist. - Decompress (read) —
node:zlibfirst (it is the only zstd implementation that takes an allocation-time output bound — see Decompression cap), then Bun’s global, then the optionalfzstdpeer-dep, so a runtime without native zstd can still read zstd bodies written elsewhere oncefzstdis installed.
The native candidates — the compressors and the two native
decompressors alike — are picked by calling them, not by checking
that the function exists; the two differ on Deno, whose node:zlib
exports zstd entry points with no binding behind them. So a runtime
without working zstd gets the framework’s own message naming the
missing capability, and the fzstd fallback below them stays
reachable. The fzstd rung is the one that is not call-checked,
deliberately: it is pure JavaScript with no native binding that could
be missing, so a successful import already answers what the call
would ask.
Those two bullets read as independent and are not, so the consequence
is worth stating outright: a runtime without native zstd can read
zstd bodies, but it cannot be configured for zstd. Choosing an
algorithm expresses write intent, so registration probes the compress
path — which means a Deno replica configured zstd to match the Bun
writers it follows fails to start. Configure such a reader with
gzip (or none) instead: the configured algorithm decides only what
the store writes, while every read decodes per the body’s own
ATS1 flag, so a gzip-configured store reads its peers’ zstd bodies
unchanged.
gzip uses node:zlib and works everywhere (Bun, Node, Deno) with
no extra dependency.
Both algorithms call the runtime’s async forms — zlib.gzip, not
gzipSync; zlib.zstdCompress, not zstdCompressSync — so a compressor
called from inside an actor’s turn does not hold the thread every other
actor runs on. Where that CPU actually lands differs by runtime; see
CPU cost.
How it works
Section titled “How it works”On put: serialize value → JSON bytes → compress(bytes) → [encrypt] → frame with ATS1 manifest → backend.put(framed)
On get: backend.get → framed bytes → read ATS1 manifest → [decrypt] → decompress per the manifest's algorithm flag → JSON → deserializeEvery body carries a small ATS1 manifest header; a 2-bit
field in its flags byte records the compression algorithm
(none / gzip / zstd). Decompression is driven entirely by
that flag — not by any HTTP header. (On S3 the framework also
sets Content-Encoding as a courtesy marker, but it is never the
source of truth for decoding.)
Because each body self-describes its algorithm, mixing none /
gzip / zstd bodies in the same bucket works — every object
decodes per its own manifest.
Decompression cap (bomb protection)
Section titled “Decompression cap (bomb protection)”Decompression on read is bounded to 512 MiB of output by default, so a tampered or hostile compressed body (a few KB that inflates to many GB) can’t exhaust memory during recovery (security audit #3).
gzip and zstd hand the bound to the decoder via zlib’s
maxOutputLength, so an over-cap body is refused before its output is
allocated — the bomb never gets the memory it was built to claim. Every
path additionally asserts the decoded size, which is what bounds the two
that cannot take the option (Bun’s Bun.zstdDecompress, reached only
on a Bun without zstd in node:zlib, and the fzstd fallback). There the
cap is still enforced, but only after the output exists — so on a runtime
that falls back to either, treat the bound as a correctness check rather
than as bomb protection.
Either way the read fails with an error instead of OOM-ing the process, and
the error names the bound (maxOutputBytes=…) so it is clear which setting
to reach for. Snapshots / durable state are
far smaller than this in practice; if you legitimately store larger bodies,
raise the bound with withMaxDecompressedBytes on the store (or plugin)
options:
const objectStorageDurableStateStoreOptions = ObjectStorageDurableStateStoreOptions.create() .withBackend(backend) .withMaxDecompressedBytes(2 * 1024 * 1024 * 1024); // 2 GiB; Infinity opts outwithMaxDecompressedBytes is available on the snapshot-store, durable-state-store,
and plugin options; the value is validated at construction (a non-positive /
non-integer cap throws OptionsError).
Changing the level
Section titled “Changing the level”The level is an encoder-only setting. It is not stored on the wire (the manifest records the algorithm, not the level), and the decompressor never needs it — gzip and zstd frames are self-describing.
So changing the configured level requires no migration: bodies written at the old level keep decoding, new bodies use the new level, and the two mix freely. You can also raise or lower the level per-actor at any time without touching existing data.
Compression + serialization
Section titled “Compression + serialization”serialize → JSON bytes → compress → [encrypt] → storeCompression runs after serialization and before encryption
— compressing ciphertext is pointless (it’s high-entropy), and the
fixed order also avoids CRIME-style side channels. For maximum
size reduction on large text-heavy state, zstd at a higher level
is usually the best single knob to turn.
When to use each
Section titled “When to use each”| Scenario | Algorithm |
|---|---|
| Large text-heavy state (big JSON, long descriptions) | zstd |
| Mixed text + numbers | zstd or gzip |
| Maximum portability, no native zstd guaranteed | gzip |
| Already-encrypted / already-compressed bytes | none (no benefit) |
| Storage-cost-bounded, write-once-read-many | zstd (level 15-19) |
| CPU-bounded write path | gzip level 1 or zstd level 1 |
CPU cost (rough guidance)
Section titled “CPU cost (rough guidance)”Single thread, ~100 KB blob — order-of-magnitude only; real numbers depend on payload and runtime:
- gzip level 6 — ~1-3 ms encode, ~0.5 ms decode.
- gzip level 1 — ~0.5 ms encode, fast decode.
- zstd level 3 (default) — encode comparable to gzip, better ratio, very fast decode.
- zstd level 19 — noticeably slower encode, decode still fast.
- zstd levels 20-22 (ultra) — much slower encode for marginal gains; reserve for write-once-read-many bulk data.
zstd’s asymmetry (cheap decode at every level) makes it a strong default for read-heavy state. For typical state-store workloads (small objects, modest write rate) the CPU cost is invisible.
Where that CPU lands differs by runtime, and it matters more than
how much of it there is. Compression is called from inside an actor’s
turn, and the framework calls the runtime’s async zlib forms
(zlib.gzip, zlib.zstdCompress, …) rather than the *Sync ones. On
Bun and Node the deflate then runs on the libuv thread pool and the
actor system keeps running: measured on a 24 MiB body, a 1 ms timer kept
firing 30–115 times during the call, where the sync forms let it fire
zero times. On Deno 2.6 the async forms defer the call and then run
it as one main-thread block — correct, but no more live than gzipSync
— so on Deno a large body is tier-2 work for the
OffloadPool; the persistence
layer deliberately does not route through it on its own. The pure-JS
fzstd read fallback has no async form and decodes on the main thread on
every runtime.
The trade the async forms make is throughput for liveness: the streaming
path costs more per call — on Node an async gunzip of the same body ran
two to three times longer than the sync one — while the event loop stays
free for the whole of it. For a state store that is the right side of the
trade: the extra milliseconds land on the one actor that asked for the
write, not on the p99 of every actor in the process.
Per-actor override
Section titled “Per-actor override”class SmallStateActor extends DurableStateActor<...> { protected compression() { return { algorithm: 'none' as const }; }}
class LogStateActor extends DurableStateActor<...> { protected compression() { return { algorithm: 'zstd' as const, level: 19 }; }}Override the store-level default per actor. Useful when:
- Most actors have small state (no compression needed).
- A few have large text-heavy state (high compression).
See Per-actor policies.
Where to next
Section titled “Where to next”- Object storage overview — the bigger picture.
- Encryption — the complementary at-rest feature.
- Per-actor policies — configuring compression per-actor.
