Hutter enwik9
DUMBer V1G · DU3 structural codec
Structure first.
Entropy later.
A reversible structural representation built around bounded local IDs, deterministic global reconciliation and exact reconstruction.
best isolated run
best isolated run
Exact receipts · SHA-verified reconstruction · measured on PC2 NVMe
01 / SCALE
Measured from kilobytes to 100 GB.
Full structural encode includes DU encode, dictionary merge and gmap. Small-system microbenchmarks are separated from large-corpus receipts.
| Input | Corpus | Structural encode | Structural restore | Exact |
|---|---|---|---|---|
| 100 KB | Hutter enwik5 | 8.893 ms | 2.277 ms | PASS |
| 1 MB | Hutter enwik6 | 44.567 ms | 9.563 ms | PASS |
| 10 MB | Hutter enwik7 | 320 ms | 40 ms | PASS |
| 100 MB | Hutter enwik8 | 1.45 s | 180 ms | PASS |
| 1 GB | Hutter enwik9 | 10.67 s | 1.18 s | PASS |
| 10 GB | OKC extended | 137.29 s | 12.84 s | PASS |
| 100 GB | OKC extended | 26m 03.86s | 3m 38.47s | PASS |
10 GB is SHA-verified as the first 10,000,000,000 bytes of the measured 100 GB extended corpus. 100 KB / 1 MB microbench values are medians from 50 CLI invocations collected under concurrent system load.
Pick the constraint.
Keep the receipts.
WORKLOAD
Hutter enwik9 · 1 GB
ENCODE + TRANSFER
Network-aware
raw zstd19
—
—
DU3 + zstd19
—
—
BYTES —
WALL —
NO SYNTHETIC SCORE
Your effective link
10 Mbps
Current transport comparisons use zstd as a widely available reference entropy backend. Entropy coding is pluggable and is not part of DU3. Views expose different optimization dimensions rather than a single synthetic score.
Open a file.
See what the machine sees.
LOCAL ONLY
Drop a PDF, Markdown or text file.
Nothing is uploaded. The browser reads the bytes locally. Today: byte inspector. Next: DU3 V1G structural roundtrip in WASM.
waiting for input
IDLE
bytes—
read—
sha-256—
SHA-256
—
HEX / first 256 bytes
visual loader v0
00 :: feed me bytes
NEXT / DU3 V1G WASM
file → structure → restore → SHA-256 PASS
Structural transform only. No zstd, Rare1 or zRank in the browser demo. The point is to expose what downstream stages receive, measure retained representation bytes, and prove exact local recovery.
The browser demo will use the already-public DU lineage as the exposure boundary, while current bounded reconciliation and experimental model work remain outside the client bundle.
Small sharp things.
DU3 / WASM
Run the structural transform locally, inspect what downstream coders receive, then restore the original bytes and verify SHA-256.
Rare1
Optional long-tail transform under active research. Kept as an explicit pipe, separate from the DU3 baseline.
Open module →zRank
Experimental model transform behind an explicit pipe boundary. Independent from DU3 and from the entropy backend.
Open module →Bounded reconciliation
The next memory target: merge and gmap32 reached roughly 13.2 GiB and 16.8 GiB RSS at 100 GB.
05 / RESEARCH
The strange shelf stays open.
Receipts, working notes, experimental codecs and older mechanisms stay searchable. DU3 is the active structural baseline; Rare1 and zRank remain optional research modules, not hidden dependencies.
DOE
Older experimental work preserved as part of the research trail.
Explore →Pongo
Research artifact from the wider OKC codec laboratory.
Explore →ZXSynth
Synthetic and structural experiments from the archive.
Explore →OCK Rotor
Earlier codec machinery, preserved for reference and reuse.
Explore →
06 / BUILD
Your data has structure.
Your data has structure.
Use it.
Need a custom structural codec, model-facing transform, data-plane experiment or integration? Built in Galicia; designed for local compute, storage and edge workloads.