A separate model, a separate question
The language experiment places byte positions on a causal line. Each position begins with a learned byte embedding and updates using its own state and information from positions to its left. It does not share the memory experiment’s associative store or periodic ring.
This is an attention-free neural sequence model, not a transformer and not a combined online-learning chatbot. Its purpose here is to establish a small, inspectable implementation and a controlled starting point for further research.
Inside the checkpoint
The supplied model has 29,056 parameters, 32 state channels per byte, a hidden width of 64, a left-neighbour radius of two and eight shared update iterations. A fixed anchor derived from the byte embedding accompanies each update. A sigmoid gate mixes the previous state with a tanh candidate.
s′ = (1 − gate(h)) s + gate(h) tanh(candidate(h))
The same learned update is reused at each iteration. A final linear head maps the state to 256 next-byte logits.
Causality imposes a real limit
With radius two and eight updates, the maximum dependency window is 1 + 2 × 8 = 17 bytes. This is a small receptive field, not an unbounded conversational memory. A long input does not let this checkpoint reason over its whole history.
The streaming implementation keeps a separate predecessor cache for every update stage. Reusing only final states at all stages would not reproduce the original full forward pass. The browser port follows the per-stage protocol and is checked against native PyTorch logits.
What it was trained on
The corpus is original template-generated English built from combinations of colours, actors, verbs and objects. There are 4,096 combinations: 3,584 training sentences and 512 held-out combinations. Vocabulary and short substrings overlap between the splits.
The longer supplied run used 2,000 updates and recorded 0.4644 sampled validation bits per byte. It was a single-seed experiment with a narrow corpus, not an instruction-following, natural-language or reasoning benchmark.
Sampling changes the experience
At a low temperature, high-probability template continuations dominate. At higher temperatures, the same checkpoint can produce malformed words, repetition and incomplete sentences. The lab exposes temperature rather than filtering out those failures.
Browser sampling uses its own documented seeded generator, not PyTorch’s multinomial random stream. Numerical parity is checked at the logits and argmax level; matching a particular published sampled paragraph is not the test.
What happens in your browser
Python executes in ZIPP WebAssembly. Dense matrix operations are submitted through ZIPP’s validated graph protocol to WebGL2, or to the explicitly selected CPU reference backend. Python performs nonlinearities, sampling and stage-cache bookkeeping.
That is a hybrid browser inference implementation—not full PyTorch running in WASM, and not training on your prompt. The checkpoint stays frozen. Try it in the causal byte lab.