PSSA reads text one token at a time through a recurrent state-space layer, keeps a bank of episodic memories it can look things up in, and rewrites part of its own weights while it runs. Written in Rust from scratch: no PyTorch, no TensorFlow, no ML framework underneath.
A transformer scores every pair of tokens in the context, so its cost per step grows with the square of sequence length and the whole context is re-read at every step. PSSA carries one fixed-size state along the sequence in a single left-to-right pass, and looks things up in a memory bank instead of re-reading the context, so cost grows linearly with length.
| PSSA | Transformer | |
|---|---|---|
| Training cross-entropy | 3.98 | 4.43 |
| Held-out cross-entropy | 3.997 | 4.429 |
| Held-out perplexity | 54.4 | 83.8 |
| Generation speed, same CPU | ~12× | 1× |
Source, training scripts, and the full README with benchmark tables live in the repository. Checkpoints are on Hugging Face.