Zenodo preprint
Adaptive column models for structured data.
This paper is deposited on Zenodo. It has not been submitted to arXiv and is not a peer-reviewed publication.
Research / evidence
Inspect the work.
Keep the context.
The paper explains the models and the choices. The evidence shows where they help, where they lose, and what remains unproven.
Reproducible measurements
Follow a result
to its source.
Current comparisons link to source provenance, measured settings, raw CSV rows and captured source/build identities.
Preprint · Zenodo · 23 October 2026
PIZD: Adaptive Column Models and Structural Factorization for Lossless Compression of Logs and Structured Data
Dany Cohen’s paper describes PIZD’s structural and column models, coding paths, and measured trade-offs. The Zenodo record preserves this preprint and its citation metadata. It has not been submitted to arXiv and is not a peer-reviewed publication. Its reported experiments concern PIZD 6.0.0-rc8 and should be read separately from the current website benchmark panel.
What the current panel
can tell you.
A controlled comparison of specific presets on these excerpts. It cannot establish a universal ranking, behavior on every production workload or independent replication.
Public inputs
16 LogHub sources, 2 historical HTTP windows, 4 GH Archive event windows and 8 public parser fixtures. Capture provenance varies. Some families were already used during development.
Fair accounting
The same bytes are restored. Dictionaries are trained on later disjoint windows and fully charged per individual file. Separate raw and trained-dictionary comparisons are selectable.
All outcomes
Small archives can cost more CPU. Some models lose. The raw reports preserve poor results, timing variation and earlier rejected approaches.
Corpus and source identities
Corpus manifest →Byte ranges, source citations, training windows and limitations.
Captured build
Run manifest →Version, source hashes, worker identity, host and measurement configuration.
Development archive
Historical experiments retain their original version and scope. They are not the current release scoreboard.
Full pipeline autopsy
Inspect →Thread timelines, dependency costs, failed candidates and the isolated budget-pruning experiment.
Production integration verification
Inspect →Actual CLI archives, worker identity checks and superseded-build handling.
Bounded selector investigation
Inspect →A rejected heuristic: modest size savings, extra CPU and wrong choices.
Decoder arenas
Inspect →Compatible dictionary and integer-decoder allocation work.
Complete repository evidence
Browse on GitHub ↗Additional corpus studies, historical reports, test tooling and experiments.
Measured version and source version
One result, one source snapshot.
The interactive public comparison is the frozen 6.0.0-rc1 run; its version is retained. The separate 18-input generated panel measures 6.0.0-rc9. Neither is a fresh comparison of the isolated rc9 candidate.
The rc9 density candidate reduces public30 Native thin/template archives from 687,537 to 657,356 bytes (4.39%), but verified in-process decoding takes 5.778 times longer in its paired audit. This is a separate candidate pending integration review, not a universal speed gain. Familiar inputs and same-family later windows do not establish independent generalization.
The engineering contribution combines exact-layout models, exception-aware field dependencies and complete-cost representation choices. These build on established compression methods. No matched comparison with specialized log compressors or new entropy-coding theorem is claimed.
Publication claim audit → · Candidate sizes, CPU costs and source identities →
Historical rc5 study · separate research frames
Measure where the filter belongs.
528 complete alternate frames test actual Native entropy channels. A fixed context/MTF model set reduces the public30 archive total by3.63%; additional models reach an offline4.16% reduction. These research frames are not integrated into the CLI. The accepted rc5 release separately passes295 tests and972 matched restores; the18-input enterprise benchmark has been rerun. Historical public scoreboard values retain their measured version.