PIZD / Practical guides

Log compression / practical evaluation

Compare PIZD, Zstd, gzip and Brotli on your logs

Measure stored bytes, compression time and verified decompression using a repeatable local comparison and clearly identified presets.

Updated October 27, 2026 · PIZD 6.0.0-rc9 · standalone CLI workflow

“Smaller” and “faster” answer different questions. A cold archive may justify extra encoding time to save storage, while a busy ingestion pipeline may prefer faster compression. Use the same inputs and explicit settings to decide whether PIZD improves your actual workload.

Record the input and environment

Start with representative logs you can process locally. Include small files, ordinary windows and difficult or irregular records. Record input byte counts, PIZD version, machine, CPU, available memory and the exact command. Avoid uploading customer logs to a public comparison. Keep already-compressed files separate from structured-data results.

Run the local benchmark

pizd bench access.log --iterations 5 --export log-comparison.md

The benchmark command compares the configured PIZD and baseline codec paths on the same input. Keep its exported report rather than copying only the most favorable number. The local benchmark’s presets are identified in its output; do not assume they are identical to every preset in the published website panel.

Read the important columns together

Compare codec versions and presets explicitly. A high-density PIZD profile against a fast baseline answers a different question from a matched density comparison. For dictionary-based modes, state whether the dictionary is shared, how it was trained and how its bytes are charged.

Use the published panel as a checkable example

The linked 6.0.0-rc9 public panel contains 30 bounded excerpts totaling 11,815,190 input bytes. Hybrid thin L9 stores 687,206 bytes, versus 835,071 bytes for Zstd L9: 17.7% fewer stored bytes in that panel. Its summed per-file median encoding time is 548.462 ms versus 252.232 ms; verified decoding is 35.621 ms versus 11.787 ms. That means about 2.17× encoding time and 3.02× verified decoding time.

These are totals across separately compressed files, not one monolithic archive or an end-to-end I/O benchmark. All 540 configuration rows in the report pass SHA-256 verification. Download the raw CSV and inspect the run manifest.

Retain the losses and define success

Look at each file, not only the total. A size gain may disappear on another source or be outweighed by CPU and restore costs. The bounded public corpus does not establish independent generalization or universal superiority over Zstd, gzip, Brotli or specialized log compressors. Before changing a pipeline, define a maximum encoding cost, a restore-time budget and a minimum useful reduction on your own representative inputs.

The benchmarks page keeps the broader comparisons and methodology. Research candidates are separate from the measured CLI configuration and should not be treated as released behavior.