A model can now generate new LLM weights from training mixtures
Researchers show that a generator can produce pretrained LLM weights for held-out mixtures of web text, books, math, code, and French, giving weight-space learning a route beyond copying existing models. They trained GPT-2 populations at 51M, 124M, and 355M parameters, then conditioned a flow-matching model on each mixture. The generated models had similar downstream perplexity and capability scores to models in the zoo.