Training a coding model to paint watercolours with TRL and OpenEnv

Sergio Paniego describes an open-source reproduction of Surya Narreddi’s approach to training language models to create watercolor-style paintings by writing JavaScript with the p5.brush library, using TRL, OpenEnv, Hugging Face Jobs, Spaces, and published datasets and model artifacts. The system trains a Qwen-based model through reinforcement learning with rewards combining code validity, length, HPSv3 image-preference scores, and a vision-language pairwise judge that compares outputs with a personally curated pool of 178 model-generated hibiscus paintings. Three reward mixes showed that HPSv3 alone mainly improved reliability by reducing failed or nearly blank paintings, while pairwise judging more strongly improved paint coverage, diversity, and resemblance to the curated aesthetic. The author details engineering challenges involving learning-rate settings, LoRA targeting for a mixture-of-experts model, rendering latency, scoring failures, websocket bugs, and variability caused by randomly sampled reference images. Although the results suggest that aesthetic preferences can steer a coding model toward a watercolor style, the project emphasizes that the reference pool—and the human taste embedded in its selection—ultimately determines what the model learns to consider visually successful.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.