PIZD / Practical guides

Log compression / practical evaluation

Compress and restore logs without a dictionary server

Create a self-contained PIZD archive, restore it offline and verify that every original byte is preserved.

Updated October 27, 2026 · PIZD 6.0.0-rc9 · standalone CLI workflow

Log archives are easier to move when restoring them does not depend on a remote dictionary service. PIZD’s standalone mode carries the archive’s required information with the file. This walkthrough keeps the original input, creates an archive, restores it to a new path and compares the two files.

Prepare an input and check the build

Use a local log or JSON Lines file that you are allowed to process. These commands use access.log as the input name. Start with pizd --version and retain the version with your results. Check release availability and build instructions; download availability and the benchmarked version are separate facts.

Keep the original file until restoration and your normal backup process have been checked. PIZD’s source-available license has production-use conditions; read the license terms before adopting it in a production workflow.

Create a standalone archive

pizd compress access.log --standalone --offline --thin -o access.pizd

--standalone creates a self-contained archive. --offline selects the offline workflow rather than contacting an optional remote pattern service. --thin favors stored size and may spend more CPU time. This is a starting point for evaluating archival density, not a promise that every file becomes smaller or encodes faster.

Restore to a separate file

pizd decompress access.pizd --offline -o restored.log

The output is deliberately named differently from the input. The decoder verifies the archive’s integrity checks while restoring it. A successful decode is necessary, but an independent comparison makes the end-to-end check easy to inspect.

Compare every original byte

cmp access.log restored.log
wc -c access.log access.pizd restored.log

On macOS and Linux, a successful cmp returns exit status 0 without output. The original and restored byte counts must agree. The archive’s count is the stored-size measurement; a smaller byte count alone does not establish a complete improvement in the workload.

Choose the profile deliberately

Begin with the default profile for a balanced evaluation. Compare --fast when encoding time matters and --thin when archive size matters more. Structured logs can provide repeated layouts and values that repay modeling costs. Tiny files, irregular text and already-compressed content may not. Decode cost and memory behavior still matter even when an archive is smaller.

Understand what the archive does not provide

Lossless restoration is not encryption, access control, backup replication or an availability guarantee. Keep the input’s normal security and retention policy. A .pizd archive and a reusable pattern dictionary are different artifacts; this guide uses standalone archives precisely so the restore does not require a separate dictionary service.

For production evaluation, test truncated or corrupted copies in an isolated directory, retain known-good originals, and measure restore time on the hardware that will actually perform recovery. Follow the comparison guide before replacing an existing compression workflow.