# Opt-in factorized templates with Native — 5.0.0-rc6

Verified local implementation, unpublished. The historical directory suffix identifies the rc5 starting point, not the measured final package version.

## Result

The existing factorized-template route now supports Native through `--template-factorization --thin --standalone`. Ordinary profiles are unchanged; a complete template archive must be strictly smaller to replace the incumbent. Both existing raw-string and integer-delta parameter representations compete under the existing Native flat/segregated coder. Native output has no Zstd/Deflate payload or external pattern dictionary. There is no new template grammar or entropy-coding invention.

| Scope | Sources | Ordinary bytes | With option | Reduction | Wins | Encode time change | Verified decode time change |
|---|---:|---:|---:|---:|---:|---:|---:|
| correlated later window | 6 | 94,891 | 94,891 | 0.00% | 0 | +21.2% | -0.3% |
| generated project sample | 14 | 210,916 | 202,642 | 3.92% | 2 | +10.2% | -0.7% |
| public development excerpt | 30 | 757,504 | 748,127 | 1.24% | 7 | +14.0% | -1.2% |

## Methods and limits

Actual CLI subprocesses: 3 warmups plus 5 measured rounds, alternating ordinary/option order. Encoding includes process launch and file I/O; verified decoding includes process launch, file I/O and the original-source cryptographic check. Exact byte comparison and an independent SHA-256 run outside timers. Aggregate times sum per-source medians. This shared desktop has background activity; raw observations and slower outcomes are retained. These are not API-only codec timings, confidence intervals or a claim of speed superiority over Zstd/Brotli.

Sources: 30 familiar bounded public excerpts, six later Spark/Windows/HealthApp windows from the same previously evaluated families, and fourteen generated development samples. None is a new independent held-out production system. The later windows do not show a new density gain; keep that result visible. No IoT/RSS or 20-GiB claim is made.

Existing eligibility remains in force: standalone density policy, no nonempty loaded pattern store, non-JSON delimited text, 512 bytes to 8 MiB, at least four records, at most 64 KiB per line and 256 parameters per record, plus the existing two-region template-diversity screen. Larger stream chunks can bypass this candidate; shared schemas across chunks and more general JSON templates remain separate work. Native decoded models/flat token streams must fit the existing reader's original-source-derived budgets.

Verification: 817 actual CLI source restores, 286 passing combined tests, 9 selected-template files read by the preserved rc5 CLI, plus SHA-256/BLAKE3 × ordinary/rANS4 compatibility controls. The existing current Native decoder reads the new opt-in frames without any grammar changes. Retained ordinary numeric archives keep their existing version requirements. Defaults are compared against frozen current-source baseline sizes on all fifty sources, not an older report whose sources differ.

The initial current-source pilot preserved both parameter models and tried the existing Native coder under two escape choices. All complete candidate archive costs are retained, including losers. Its one-shot stage timers are diagnostic and excluded from the matched CLI totals. Live integration retains the production eligibility screen, source-derived decoded-model and flat-token budgets, both checksum policies, optional rANS4 and strict ordinary tie behavior.

The separate `cli_correctness_results.json` capture preceded final timing and ran under concurrent work; its timings are not headline evidence. An intermediate release build failed because a peer allocation edit referenced a constant in the wrong module. The peer corrected it before CODEX's guarded repair; final combined source/build/test identities supersede the earlier successful Native-only debug suite. Both logs remain visible. Source changes outside Native integration are recorded in the complete combined snapshot, not attributed to Native template coding.

The measured decoder time differences are descriptive. Unchanged retained archives are controls for host/process variation; an aggregate reduction alone is not a proof that the option accelerates every decoder. No headline decoder speedup is attributed to the Native template option.

## All sources

Best trial is the smallest exhaustive pilot candidate, even when larger than ordinary PIZD. Option bytes show the actual integrated archive retained after full-cost competition. Negative trial percentages are better; positive values are losses.

| Source | Ordinary B | Best trial B | Trial vs ordinary | Actual option B | Encode ms ordinary → option | Verified decode ms ordinary → option |
|---|---:|---:|---:|---:|---:|---:|
| 01_loghub_hdfs.log | 43756 | 42835 | -2.10% | 42835 | 65.04 → 100.69 | 9.94 → 10.27 |
| 02_loghub_hadoop.log | 14204 | 15936 | +12.19% | 14204 | 32.78 → 40.50 | 8.64 → 8.58 |
| 03_loghub_spark.log | 16349 | 19000 | +16.22% | 16349 | 50.50 → 60.62 | 10.73 → 11.15 |
| 04_loghub_zookeeper.log | 6310 | 8115 | +28.61% | 6310 | 47.00 → 53.49 | 11.84 → 11.49 |
| 05_loghub_openstack.log | 41821 | 49504 | +18.37% | 41821 | 81.70 → 82.32 | 8.74 → 8.92 |
| 06_loghub_bgl.log | 13888 | 11518 | -17.07% | 11518 | 54.81 → 63.01 | 12.54 → 9.62 |
| 07_loghub_hpc.log | 16770 | 16705 | -0.39% | 16705 | 57.68 → 69.61 | 11.13 → 9.47 |
| 08_loghub_thunderbird.log | 39015 | 41564 | +6.53% | 39015 | 68.55 → 90.97 | 11.42 → 11.50 |
| 09_loghub_windows.log | 10770 | 18974 | +76.17% | 10770 | 46.39 → 58.75 | 10.41 → 10.56 |
| 10_loghub_linux.log | 31830 | 35781 | +12.41% | 31830 | 68.65 → 86.28 | 8.91 → 9.63 |
| 11_loghub_mac.log | 22240 | 24967 | +12.26% | 22240 | 57.73 → 68.22 | 8.63 → 8.57 |
| 12_loghub_android.log | 40960 | 48194 | +17.66% | 40960 | 77.55 → 77.84 | 8.90 → 9.81 |
| 13_loghub_healthapp.log | 20545 | 25737 | +25.27% | 20545 | 65.79 → 78.21 | 12.34 → 11.87 |
| 14_loghub_apache.log | 25470 | 22368 | -12.18% | 22368 | 59.41 → 75.60 | 9.31 → 9.68 |
| 15_loghub_openssh.log | 13401 | 12731 | -5.00% | 12731 | 50.65 → 70.84 | 10.48 → 9.85 |
| 16_loghub_proxifier.log | 24696 | 22874 | -7.38% | 22874 | 53.82 → 65.85 | 10.72 → 9.16 |
| 17_beats_nginx_access.log | 774 | 1091 | +40.96% | 774 | 15.04 → 22.45 | 7.13 → 7.49 |
| 18_beats_nginx_error.log | 663 | 784 | +18.25% | 663 | 13.01 → 13.70 | 6.96 → 7.01 |
| 19_beats_postgresql_log.log | 2531 | 2623 | +3.63% | 2531 | 17.30 → 18.81 | 7.29 → 7.28 |
| 20_beats_mysql_slowlog.log | 2494 | 3245 | +30.11% | 2494 | 13.36 → 14.93 | 7.20 → 7.23 |
| 21_beats_mongodb_log.log | 1784 | 2049 | +14.85% | 1784 | 14.97 → 15.20 | 6.87 → 6.79 |
| 22_beats_redis_log.log | 1505 | 1078 | -28.37% | 1078 | 45.08 → 47.60 | 9.05 → 10.34 |
| 23_beats_elasticsearch_audit.log | 1633 | 1760 | +7.78% | 1633 | 14.15 → 13.80 | 7.00 → 7.00 |
| 24_beats_traefik_access.log | 1058 | 1228 | +16.07% | 1058 | 12.70 → 13.52 | 6.67 → 6.77 |
| 25_nasa_http_jul95.log | 36151 | 76960 | +112.88% | 36151 | 76.11 → 78.09 | 8.69 → 8.91 |
| 26_nasa_http_aug95.log | 37283 | 74224 | +99.08% | 37283 | 76.88 → 75.38 | 9.04 → 8.81 |
| 27_github_events_20150101_00.jsonl | 76935 | not eligible | — | 76935 | 80.22 → 80.32 | 9.22 → 9.47 |
| 28_github_events_20150101_06.jsonl | 71978 | 79221 | +10.06% | 71978 | 81.04 → 79.51 | 9.55 → 9.06 |
| 29_github_events_20150101_12.jsonl | 68120 | 75510 | +10.85% | 68120 | 75.26 → 74.46 | 9.15 → 9.13 |
| 30_github_events_20150101_18.jsonl | 72570 | 82530 | +13.72% | 72570 | 77.36 → 76.35 | 9.41 → 9.08 |
| healthapp_1024k.log | 18949 | 32193 | +69.89% | 18949 | 66.67 → 81.82 | 11.91 → 12.03 |
| healthapp_2048k.log | 19362 | 32638 | +68.57% | 19362 | 66.93 → 81.78 | 12.00 → 12.17 |
| spark_1024k.log | 20036 | 25138 | +25.46% | 20036 | 63.26 → 76.19 | 11.16 → 10.92 |
| spark_2048k.log | 19784 | 24678 | +24.74% | 19784 | 60.11 → 76.42 | 11.30 → 11.01 |
| windows_1024k.log | 8267 | 15817 | +91.33% | 8267 | 42.30 → 43.49 | 10.42 → 10.57 |
| windows_2048k.log | 8493 | 17340 | +104.17% | 8493 | 44.07 → 56.44 | 10.64 → 10.54 |
| 01_nginx_access.log | 10853 | 17483 | +61.09% | 10853 | 52.78 → 63.53 | 7.86 → 7.91 |
| 02_nginx_error.log | 13217 | 11437 | -13.47% | 11437 | 49.81 → 56.95 | 10.61 → 8.88 |
| 03_app_json.log | 23283 | 37239 | +59.94% | 23283 | 46.00 → 45.67 | 9.39 → 9.43 |
| 04_k8s_container.log | 16773 | 23986 | +43.00% | 16773 | 40.60 → 40.69 | 9.40 → 9.34 |
| 05_syslog_systemd.log | 9389 | 17160 | +82.77% | 9389 | 44.48 → 43.01 | 7.76 → 7.82 |
| 06_aws_cloudtrail.json | 9320 | 23144 | +148.33% | 9320 | 34.67 → 34.93 | 7.97 → 8.04 |
| 07_aws_vpcflow.log | 33648 | 43619 | +29.63% | 33648 | 89.18 → 108.55 | 8.62 → 8.97 |
| 08_dns_queries.log | 27418 | 20924 | -23.69% | 20924 | 54.58 → 62.28 | 8.55 → 8.66 |
| 09_postgres_slowquery.csv | 7504 | 9336 | +24.41% | 7504 | 36.13 → 41.21 | 8.40 → 8.39 |
| 10_rest_api_batch.json | 5762 | 13633 | +136.60% | 5762 | 28.14 → 27.76 | 8.37 → 8.72 |
| 11_opentelemetry_traces.json | 7242 | 19822 | +173.71% | 7242 | 38.68 → 38.06 | 11.66 → 11.76 |
| 12_financial_ticks.csv | 21889 | 24034 | +9.80% | 21889 | 60.03 → 77.32 | 12.81 → 12.70 |
| 13_stripe_webhooks.json | 9412 | 24366 | +158.88% | 9412 | 29.60 → 29.76 | 7.84 → 7.81 |
| 17_github_actions_ci.log | 15206 | 26964 | +77.32% | 15206 | 45.86 → 47.19 | 9.41 → 9.27 |

## Reference size positions

Join only the same 30 source hashes to the fresh rc6 reference panel. Complete archive bytes are comparable here; CLI timings above must not be compared with the panel's API timings. Native remains at 16/30 size wins versus raw Zstd L9 and Brotli Q9, and 24/30 versus Gzip L9, with or without the option. The option reduces existing winners and some still-losing files; it does not gain new reference positions on this panel. Stronger levels and trained-dictionary accounting remain in their own reports.


## Usage

```sh
./target/release/pizd compress server.log --offline --standalone --engine native --thin --template-factorization -o server.pizd
./target/release/pizd decompress server.pizd --offline --strict-capabilities -o restored.log
```

Additional encode work is explicit. `--fast` and non-thin CLI combinations reject this option. Hybrid retains its existing option and all main reference codecs remain separate measurements.

## Artifacts

`inputs.json`: source hashes and baseline source identities. `size_results.json`: every complete raw/delta trial. `cli_results.json`: full paired observations and exact checks. `integrated_build.json`: measured source/binary hashes. `combined_tests.log`: full combined checks. `baseline_sources.tar.gz` and `integrated_sources.tar.gz`: original and final source snapshots. `integration.patch`: Native integration versus the original source. `old_reader.json`: preserved reader identity. Main updated comparator panel is `../public_logs_30_5.0.0-rc6/`; it measures ordinary profiles, not this extra opt-in route.