Training Qwen to Paint Watercolors with Pure RL
Surya trained Qwen 3.5 to have taste in watercolor painting. The results are stunning.
It’s purely RL:
The system is a four-step loop, run thousands of times during training.
The model receives a prompt, something like draw a peach hibiscus in watercolour, and writes a complete p5.brush JavaScript sketch. The sketch is rendered in a sandboxed Puppeteer environment, which produces a PNG. The PNG is judged against two random reference paintings sampled from a hand-rated pool, with a separate judge model picking the better watercolour. The judgment is converted into a reward signal, GRPO updates the model, and the loop runs again.