Memory beyond the convenient case
Measure what happens as key vocabularies grow, key dimensions shrink, records share a writer, or observation streams repeatedly overwrite associations. Test noisy values and controlled changes in key geometry. Report per-bit accuracy alongside whole-value recall.
A useful next control is a properly tuned recurrent baseline under matched state and compute constraints. A symbolic dictionary should remain visible as an algorithmic reference, not be omitted because it performs well.
Language with a larger causal horizon
Train for longer receptive fields rather than assuming extra inference iterations will work automatically. Compare fixed-depth, mixed-depth and curriculum-based recipes on held-out data. Keep prefix invariance and full-versus-streaming parity tests as the architecture changes.
A move from template combinations to a broader corpus requires new data documentation, baselines and evaluation. It would be a new experiment, not a retroactive upgrade to the current toy checkpoint.
Connecting memory and language carefully
A future model could combine a causal language state with an explicit adaptive-memory mechanism. It would need a defined observation interface, read/write policy, isolation between streams, and interventions proving where acquired facts are stored.
The current two checkpoints are not that integrated model. Attaching a memory visualisation to the language interface would not, by itself, demonstrate a language model learning while it generates.
Better browser execution
Measure the cost of Python graph construction, host transfers, shader dispatch and readback separately. Keep frequent operations on the device when the runtime supports it, and compare against a CPU reference on the same task.
The present browser port prioritises an inspectable path through ZIPP and numerical agreement. It is not an optimised GPU training engine. Hardware-driver coverage and sustained-session testing are practical next validation steps.
How to contribute
Useful contributions include a minimal reproducible issue, a numerical parity test, a carefully controlled experiment or an implementation improvement with an honest before/after measurement. The research repository is the source of record; the lab reproduction guide explains the browser and native paths.