# Native residual models and filter order

Measured 2026-10-23. Research only: no new entropy format is integrated into the CLI. Baseline is frozen Native rc3 `--thin --template-factorization --offline --standalone`.

## What is held constant

Recode only the first entropy channel of actual complete Native archives. Preserve every other archive byte, including checksum, copy/run/length channels, flags and transforms. Charge all model tables, stream lengths and 25 additional framing bytes. The inverse restores the byte-identical original archive, then the frozen CLI restores the source and its external SHA-256 is checked.

54 inputs cover 30 public excerpts plus small/medium/large controls from the filter matrix. They include nested public prefixes, generated JSON/CSV/application logs, random bytes and explicitly synthetic random-case controls. They are not 54 independent production systems. Phase1 has 270 complete alternative frames; phase2 has258. All baseline sources, including unsupported formats, were independently decoded in phase2. [Archive ledger](archive_ledger.json).

The first channel can be a literal stream or secondary LZ tokens in split mode2. Only PZSG1 receives reconstructed-model contexts:28/54 inputs,17/30 public. The independent production SegregatedCodec decoder confirms each traced model byte-for-byte. This is the inner transformed model, not necessarily the original text or a field-aware context. LZ copies are replayed causally; no future source bytes are used.

Unsupported archive formats keep the incumbent explicitly. Entropy-symbol models support30/54 inputs,18/30 public. No unseen-format gain is implied.

## Public30 results

Baseline: **722,532 B**. Fixed phase1 model set (context4, MTF) plus incumbent: **696,292 B**, 3.63% smaller. Offline combined minimum over all examined models: **692,449 B**, 4.16% smaller. This combined minimum was selected after examining complete outcomes; it is not a demonstrated cheap routing policy. Selection requires candidate exploration, whose cost is not free.

Each row below applies one fixed method with incumbent fallback. The final column forces that method on its supported subset and exposes losses that fallback hides. Positive means smaller.

| Method | Supported | Wins /30 | Total with fallback | Saving | Forced saving on supported inputs |
|---|---:|---:|---:|---:|---:|
| context4 | 18/30 | 17 | 700,606 | +3.03% | +3.30% |
| delta_huffman | 18/30 | 0 | 722,532 | +0.00% | -8.18% |
| xor_huffman | 18/30 | 0 | 722,532 | +0.00% | -6.14% |
| mtf_huffman | 18/30 | 13 | 705,409 | +2.37% | +2.37% |
| case_dense | 18/30 | 1 | 722,496 | +0.00% | -0.26% |
| delta_context4 | 18/30 | 0 | 722,532 | +0.00% | -4.31% |
| xor_context4 | 18/30 | 0 | 722,532 | +0.00% | -3.42% |
| case_then_context4 | 18/30 | 17 | 702,649 | +2.75% | +2.98% |
| context4_then_case | 18/30 | 17 | 705,242 | +2.39% | +2.58% |
| first_after_copy_context | 17/30 | 16 | 702,408 | +2.79% | +3.38% |
| reconstructed_model_context | 17/30 | 16 | 700,819 | +3.01% | +3.65% |
| last_copy_xor_context | 17/30 | 4 | 720,855 | +0.23% | -2.58% |
| last_copy_delta_context | 17/30 | 2 | 721,875 | +0.09% | -4.86% |
| first_after_copy_xor_context | 17/30 | 14 | 709,084 | +1.86% | +2.25% |
| first_after_copy_delta_context | 17/30 | 13 | 710,789 | +1.63% | +1.85% |
| context_then_local_mtf | 18/30 | 16 | 699,413 | +3.20% | +3.44% |
| mtf_then_rank_context | 18/30 | 12 | 708,623 | +1.93% | +1.66% |
| global_mtf_original_context | 18/30 | 12 | 705,153 | +2.41% | +2.34% |

## Size and source sensitivity

| Input | Source bytes | Native | Phase1 fixed set | All examined models |
|---|---:|---:|---:|---:|
| 01_loghub_hdfs.log | 524,198 | 33,925 | 31,648 | 31,285 |
| 02_loghub_hadoop.log | 239,767 | 14,204 | 14,204 | 14,204 |
| 03_loghub_spark.log | 524,190 | 16,310 | 16,310 | 16,310 |
| 04_loghub_zookeeper.log | 524,285 | 6,287 | 6,287 | 6,287 |
| 05_loghub_openstack.log | 524,234 | 41,345 | 39,538 | 39,020 |
| 06_loghub_bgl.log | 524,227 | 11,515 | 11,515 | 11,515 |
| 07_loghub_hpc.log | 524,244 | 15,918 | 15,212 | 15,212 |
| 08_loghub_thunderbird.log | 524,221 | 38,891 | 37,618 | 37,432 |
| 09_loghub_windows.log | 524,220 | 10,770 | 10,770 | 10,770 |
| 10_loghub_linux.log | 524,150 | 31,816 | 28,692 | 27,665 |
| 11_loghub_mac.log | 524,234 | 21,949 | 20,633 | 20,513 |
| 12_loghub_android.log | 524,276 | 40,854 | 38,689 | 38,493 |
| 13_loghub_healthapp.log | 524,254 | 16,629 | 16,229 | 16,229 |
| 14_loghub_apache.log | 524,233 | 22,202 | 21,617 | 21,612 |
| 15_loghub_openssh.log | 524,222 | 12,232 | 11,654 | 11,654 |
| 16_loghub_proxifier.log | 524,224 | 20,081 | 19,898 | 19,898 |
| 17_beats_nginx_access.log | 3,177 | 774 | 774 | 774 |
| 18_beats_nginx_error.log | 1,612 | 663 | 663 | 663 |
| 19_beats_postgresql_log.log | 23,312 | 2,531 | 2,531 | 2,531 |
| 20_beats_mysql_slowlog.log | 8,210 | 2,494 | 2,494 | 2,494 |
| 21_beats_mongodb_log.log | 4,502 | 1,784 | 1,784 | 1,784 |
| 22_beats_redis_log.log | 524,271 | 1,078 | 1,078 | 1,078 |
| 23_beats_elasticsearch_audit.log | 4,945 | 1,633 | 1,633 | 1,633 |
| 24_beats_traefik_access.log | 1,665 | 1,058 | 1,058 | 1,058 |
| 25_nasa_http_jul95.log | 524,244 | 36,133 | 33,195 | 33,047 |
| 26_nasa_http_aug95.log | 524,228 | 37,264 | 34,331 | 34,039 |
| 27_github_events_20150101_00.jsonl | 523,760 | 74,188 | 72,695 | 72,361 |
| 28_github_events_20150101_06.jsonl | 523,728 | 70,282 | 68,804 | 68,804 |
| 29_github_events_20150101_12.jsonl | 520,527 | 67,171 | 65,858 | 65,574 |
| 30_github_events_20150101_18.jsonl | 523,830 | 70,551 | 68,880 | 68,510 |
| nasa_large.log | 4,194,244 | 248,062 | 224,399 | 222,478 |
| hdfs_large.log | 4,194,199 | 191,658 | 176,551 | 170,483 |
| github_large.jsonl | 4,193,613 | 487,150 | 477,435 | 477,435 |
| json_large.json | 4,194,198 | 236,303 | 236,303 | 236,097 |
| csv_large.csv | 4,194,288 | 361,407 | 361,407 | 361,407 |
| app_large.log | 4,194,257 | 245,034 | 245,034 | 245,034 |
| nasa_small.log | 8,162 | 1,770 | 1,770 | 1,770 |
| hdfs_small.log | 8,065 | 1,391 | 1,391 | 1,391 |
| github_small.jsonl | 7,020 | 2,484 | 2,484 | 2,484 |
| json_small.json | 8,123 | 1,101 | 1,101 | 1,101 |
| csv_small.csv | 8,182 | 1,047 | 1,047 | 1,047 |
| app_small.log | 8,156 | 1,104 | 1,104 | 1,104 |
| random_small.bin | 8,192 | 8,247 | 8,247 | 8,247 |
| nasa_medium.log | 262,039 | 19,832 | 18,283 | 18,283 |
| hdfs_medium.log | 262,141 | 20,994 | 19,779 | 19,645 |
| github_medium.jsonl | 256,332 | 43,421 | 42,513 | 42,320 |
| json_medium.json | 262,047 | 15,531 | 15,531 | 15,531 |
| csv_medium.csv | 262,113 | 20,462 | 20,462 | 20,462 |
| app_medium.log | 262,127 | 18,292 | 17,851 | 17,851 |
| random_medium.bin | 262,144 | 262,199 | 262,199 | 262,199 |
| random_large.bin | 4,194,304 | 4,194,359 | 4,194,359 | 4,194,359 |
| mixed_case_control_small.log | 8,209 | 2,857 | 2,857 | 2,694 |
| mixed_case_control_medium.log | 262,173 | 38,817 | 37,940 | 37,202 |
| mixed_case_control_large.log | 4,194,351 | 682,479 | 682,479 | 682,479 |

## Follow-up windows and placement control

- Phase1 context4/MTF: fixed allowed modes before this follow-up, 13 further LF-aligned approximately1MiB windows; 13/13 wins, 1,439,140 ->1,390,271 B (3.40% smaller). Parent systems remain NASA/HDFS/GitHub, so this is additional-window evidence, not independent-project validation. Phase2 uses different offsets from phase1. Full provenance and hashes: [native_residual_6.0.0-rc3 windows](../native_residual_6.0.0-rc3/validation_windows.json).
- Phase2 reconstructed-model context / context-then-local-MTF: fixed allowed modes before this follow-up, 10 further LF-aligned approximately1MiB windows; 10/10 wins, 1,047,071 ->1,015,301 B (3.03% smaller). Parent systems remain NASA/HDFS/GitHub, so this is additional-window evidence, not independent-project validation. Phase2 uses different offsets from phase1. Full provenance and hashes: [native_residual_phase2_6.0.0-rc3 windows](../native_residual_phase2_6.0.0-rc3/validation_windows.json).
- Whole-file MTF and MTF→Huffman before actual Native were also tested on all30 public inputs:60 fully restored frames, zero wins. This contrasts with useful MTF placement inside the entropy channel. It does not rule out all BWT/MTF designs.

## Interpretation and timing limits

- Context-before-local-MTF improves several public logs. Reconstructed-model context improves some GitHub and Thunderbird cases beyond entropy-symbol context. Merely appending delta/XOR globally does not reproduce those gains.
- Case separation gains remain concentrated in deliberately synthetic random-case controls. Do not use them to imply ordinary logs commonly halve in size.
- The research decoder re-encodes the original Huffman channel before normal Native decoding. These scaffold encode/inverse times are not integrated production compression/decompression speed. Phase2 additionally reconstructs the inner model; whole-process CPU, model memory and routing cost require a direct decoder implementation before integration.
- These frames use PZRC0001/PZRC0002 research magic. They are not readable directly by the production CLI. A production route needs explicit format/capability signaling, bounded-memory malformed-input handling, compatibility tests and end-to-end timing. No rc5 speed or ratio claim is derived from these experiments.

[Phase1 raw results](../native_residual_6.0.0-rc3/results.json), [phase2 raw results](results.json), [all per-method summaries](method_summary.csv), [frozen source package](research_sources.tar.gz), [source identities](source_fingerprints.json).

## Overlapping-copy replay audit

On 28 actual fixed Native operation streams, bulk fill/doubling for long overlapping copies restores the same model as the byte loop. One warmup and three alternating trials give summed median stage time 3.204 ->3.059 ms (4.54% lower). Several individual cases lose; most contain little overlap. The existing RelativeEncoder already has a fast copy helper, while segregated/split reconstruction has separate loops. This local result does not establish an end-to-end win and is not integrated. [Every replay timing](copy_replay.json).