Hutter enwik9
DUMBer V1G · DU3 structural codec
Structure first.
Entropy later.
A reversible structural representation built around bounded local IDs, deterministic global reconciliation and exact reconstruction.
best isolated run
best isolated run
Exact receipts · SHA-verified reconstruction · measured on PC2 NVMe
NEW / NEREIDS PHASE 3 · 1 GB DU3 → cl100k vocabulary view · EXACT · inspect receipt →
01 / SCALE
Measured from kilobytes to 100 GB.
Full structural encode includes DU encode, dictionary merge and gmap. Small-system microbenchmarks are separated from large-corpus receipts.
| Input | Corpus | Structural encode | Structural restore | Exact |
|---|---|---|---|---|
| 100 KB | Hutter enwik5 | 8.893 ms | 2.277 ms | PASS |
| 1 MB | Hutter enwik6 | 44.567 ms | 9.563 ms | PASS |
| 10 MB | Hutter enwik7 | 320 ms | 40 ms | PASS |
| 100 MB | Hutter enwik8 | 1.45 s | 180 ms | PASS |
| 1 GB | Hutter enwik9 | 10.67 s | 1.18 s | PASS |
| 10 GB | OKC extended | 137.29 s | 12.84 s | PASS |
| 100 GB | OKC extended | 26m 03.86s | 3m 38.47s | PASS |
10 GB is SHA-verified as the first 10,000,000,000 bytes of the measured 100 GB extended corpus. 100 KB / 1 MB microbench values are medians from 50 CLI invocations collected under concurrent system load.
Pick the constraint.
Keep the receipts.
WORKLOAD
Hutter enwik9 · 1 GB
ENCODE + TRANSFER
Network-aware
raw zstd19
—
—
DU3 + zstd19
—
—
BYTES —
WALL —
NO SYNTHETIC SCORE
Your effective link
10 Mbps
Current transport comparisons use zstd as a widely available reference entropy backend. Entropy coding is pluggable and is not part of DU3. Views expose different optimization dimensions rather than a single synthetic score.
Open a file.
See what the machine sees.
LOCAL ONLY
Drop a PDF, Markdown or text file.
Nothing is uploaded. The browser reads the bytes locally. Today: byte inspector. Next: DU3 V1G structural roundtrip in WASM.
waiting for input
IDLE
bytes—
read—
sha-256—
SHA-256
—
HEX / first 256 bytes
visual loader v0
00 :: feed me bytes
NEXT / BROWSER DU
file → structure → restore → SHA-256 PASS
Structural transform only. No zstd, Rare1 or zRank in the browser demo. The browser build will track the latest released DU line: inspect the structural representation, restore the original file, and verify SHA-256.
The public DU lineage defines the browser exposure boundary. Native DU3 releases can advance independently; experimental pipeline work remains outside the client bundle.
Small sharp things.
Browser DU
Run the structural transform locally, inspect what downstream coders receive, then restore the original bytes and verify SHA-256.
Rare1
Optional long-tail transform under active research. Kept as an explicit pipe, separate from the DU3 baseline.
Open module →zRank
Experimental model transform behind an explicit pipe boundary. Independent from DU3 and from the entropy backend.
Open module →Nereids
Project an existing DU3 structural stream into the full cl100k_base mergeable-ID space while preserving the exact bytes. 1 GB receipt: 0 unmappable DU entries.
Open Nereids →
05 / RESEARCH
The strange shelf stays open.
Receipts, working notes, experimental codecs and older mechanisms stay searchable. DU3 is the active structural baseline; Rare1 and zRank remain optional research modules, not hidden dependencies.
DOE
Older experimental work preserved as part of the research trail.
Explore →Pongo
Research artifact from the wider OKC codec laboratory.
Explore →ZXSynth
Synthetic and structural experiments from the archive.
Explore →OCK Rotor
Earlier codec machinery, preserved for reference and reuse.
Explore →
06 / BUILD
Your data has structure.
Your data has structure.
Use it.
Need a custom structural codec, model-facing transform, data-plane experiment or integration? Built in Galicia; designed for local compute, storage and edge workloads.