I built a pile of new datasets, 3 new vintage models, a benchmark, an evaluation pipeline, experiments and a very good vintage tokenizer.

Demo play with the vintage-LLM: https://huggingface.co/spaces/croqaz/Piston-and-Prose