Fine‑tuning LLMs on exact, code‑generated symbolic world trajectories (world‑time compute) markedly improves performance on unseen worlds, especially for smaller models.
Researchers introduced a framework called OpenWorld that generates unlimited verified symbolic world models. Training LLMs on trajectories from many such worlds (world‑time compute) raised accuracy on held‑out worlds by up to 29 points for 0.5 B‑parameter models, with diminishing returns for larger models and longer reasoning chains.
You can use the OpenWorld framework to create unlimited verified world trajectories and fine‑tune a model on them.
Fine‑tune a 0.5 B LLM on 10 k OpenWorld trajectories to improve few‑step reasoning benchmarks.
Suggested exercise · not independently tested. Free access suggested; check current availability.
Before you start: Git · Python 3.9+ · Access to an LLM fine‑tuning pipeline
Clone the OpenWorld repository from the paper's URL.
Run the provided script to generate 10 k trajectories for a simple grid world.
Save the trajectories in JSONL format.
Fine‑tune your LLM using your preferred trainer with the generated data.
Check your result: The fine‑tuned model scores higher than the baseline on the same grid‑world test set.
Tags
Sources
Also today