250M LLM in 60 MB hits 400 tok/s on a laptop CPU
One developer trained a 250M-parameter model on 30B tokens of FineWeb and quantized the whole thing under 2 bits per weight: 60 MB, about 80 MB of RAM, 400 tok/s on a laptop CPU. The clever part is context. Beyond 2,048 live tokens, history is compressed to 320 bytes per token on disk, up to 100M tokens, and the model was trained from step one to retrieve facts from that archive. It finds answers; it just can't reason across them.