AI World Gen: thousands of tiny decisions, and a loop that graded them

A model that decides instead of writes

AI World Gen started at the AI Hackathon Barcelona 2026. The idea: describe a place, and watch an AI build its map one tile at a time. The interesting question was never "can a model draw a map". It was: given what is around this tile, what goes here? That question is asked hundreds of times per map. Asking a chat model each time is slow, expensive, and turns a one-word answer into text you have to parse.

Jev, made by TypeSafe and served through OpenRouter's decisions endpoint, is a different kind of model. You send it a state (a JSON object describing the situation) and a typed question: pick one of these options, or place this on a scale. It answers with a choice, a confidence and a probability for every option. It does not write text at all. It is built for speed (TypeSafe quotes tens of milliseconds for the model itself) and it is priced at about $0.04 per million input tokens, roughly fifty times less than the chat models I would otherwise have used for the same call. The exact bill is a section of its own below.

So the generator splits the work in two. One creative call goes to a language model, once per world: it invents the vocabulary, the six to twenty-four kinds of tile a "space station" or a "haunted mansion" is made of, with their rules. Then one decision goes to Jev, once per tile, with the vocabulary as the options and the map so far as the state. A preset world skips the creative call entirely, because the vocabulary ships with the page.

The numbers make the design possible. The test run I describe below draws 38 maps, 3,008 decisions in total. It costs about 24 cents and takes under ten minutes. A chat model in the same seat would cost dollars and take the better part of an hour, for an answer that has to be parsed back into one of the options. Cheap and fast is not a nicety here; it is what lets you run the whole thing before every change.

Version one was a green blur

The first version worked, in the sense that every call succeeded and a map came out. It was also useless. Thirteen of the fifteen maps in the first test set used one tile type for more than 85% of the cells: a village that is all grass, a station that is all corridor. The reason is obvious once you see it. A model that decides one cell from its neighbours, and whose neighbours are all grass, picks grass. Every correct local decision adds up to a wrong map.

I could have looked at three maps, changed the prompt, looked at three more, and felt good. I have done that before and I know how it ends: the change that fixes the village breaks the station, and nobody notices for a week. So before touching the generator, I built a way to measure it.

Measuring instead of looking

The measurement lives in Galtea, an evaluation platform for AI products (where I work, so take this paragraph as a demonstration of the tool as much as a recommendation). The shape of it is simple. A product has versions. A version is judged against written specifications: sentences about what a good map is. Each specification has metrics that score a map from 0 to 1, and a dataset of test cases. A test case here is one seed: a preset, a grid size, a fill order and a random seed, so every version draws the same 38 worlds.

Almost every metric is plain code in the project, not a model. "Doors sit in walls" is a loop over the door tiles. "One walkable region" is a flood fill. "The route reaches the edge" is a search. The one thing code cannot answer, "does this look like a village?", goes to a judge model, GPT-5.2, which reads the map as text and scores it. The specifications grew as the failures showed up: three at the start (barriers form structures, paths form routes, everything walkable is reachable), a fourth after version 4 ("a place reads as a place": doors in walls, walls as outlines, an enclosed room, ground in patches), and three more after version 7 (the vocabulary's own rules hold, routes lead to doors and off the map, the landmarks are there once and nothing floods the map). By the end there were 28 computed metrics and one judge.

Eleven versions in twenty hours

Version 1 was scored at 14:20 on 19 September. Version 11 at 09:49 the next morning. Each chart below is one specification, and each point is the mean score of that version over the same 38 seeds. The dotted grey chart is the judge. The numbers are in the table at the end.

The judge, which never saw a metric, agrees with the code: it moves from 0.09 on version 1 to 0.69 on version 11, and its two big steps land where the computed ones do, at the balance sheet (v2) and at the blueprint (v8). The story the charts tell, in order:

v2 · confetti

A balance sheet in the state, so the model knows which types are under-used. Coverage of the vocabulary went from 0.15 to 0.91, and the model placed the needed types anywhere: single walls, single path cells, nothing joined up.

v3 · floods

Continuation hints: "this wall line reaches you, continue it". The model did, across the whole map. 58 corridor cells in one 8 by 8 station.

v4 · lines

Code judges the one suggested continuation: short lines, closable gaps, never an over-used type. Walls form lines and closed shapes. But 21 of 23 doors stood in open ground, and no map had a room you could enter.

v5 · the model reads the sketch, a little

The whole map goes into the state, one letter per cell. Coherence moved from 0.50 to 0.53. The 16 by 16 maps showed the limit: walls still came as solid slabs, because a model deciding one cell cannot see where a rectangle should end.

v6 · less noise

The sampling had been overriding the model's own choice on 27% of the cells. Raising the floor to a third of the best option cut that to 18%: ground in patches 0.25 to 0.52, and the rare types lost their chances (coverage 0.72 to 0.54).

v7 · the local rules hold, the rooms do not

Typed placement rules (never next to X, only next to Y, on this edge) enforced in code before the question is asked. "Rules hold" went from 0.61 to 0.98. Nothing else moved: "never next to" cannot say "in a wall".

v8 · rooms

The blueprint. Code places each structure as a rectangle with a door before any cell is asked about, and each cell is only offered the types of its part: wall, door, interior, outside. Doors in walls 0.15 to 0.96; an enclosed room on every map; "a place reads as a place" 0.52 to 0.89. The price: coverage down to 0.52.

v9 · the world remembers what it lacks

The state names the things the world still misses; a unique landmark is out once placed; a route is suggested outside a planned door. Landmarks single 0.68 to 1.00, doors with a route 0.56 to 0.88.

v10 · the cap that could not fire

A ground type past twice its target is no longer offered. It changed nothing, because the maps that flooded were the settings whose vocabulary made the route the only floor: a corridor was the station's ground.

v11 · a floor that is not a road

The station gets an open deck, the city block a plaza, and corridor and street become lines. The station's route share fell from 0.53 to 0.12 of the map. The corridors are now short stubs on the deck, which is the next problem.

Regressions, and what the loop caught

Three things I would not have learnt from looking at pictures.

A new metric rewrites history. Version 1 scored 0.99 on "paths form routes", because a map that is all corridor has no isolated path cell. When the path-share metric arrived later and every old version was rescored, v1 fell to 0.74. A version table is only comparable if every row is scored with the same rules, so every new metric was run back over every old map, and every new test seed was drawn again with each old version's own code. Read the charts knowing that; the scores are today's rules applied to yesterday's maps.

Every fix costs something, and the loop shows you the bill. The sampling floor in v6 made calmer maps and starved the rare types. The blueprint in v8 gave the map rooms and took a third of its vocabulary coverage. Version 11 fixed the flooded station and turned its corridors into stubs: path continuity fell from 0.73 to 0.58 in the same run that raised path share from 0.65 to 0.84. None of these were visible in the one map I would have looked at. All of them were visible in the mean over 38.

A per-cell model cannot draw a rectangle. Seven versions tried to get rooms out of the prompt: more context, the whole map as text, softer sampling, typed rules. Doors stayed in fields. The eighth version stopped asking. Code places the buildings, the way a person drawing a map decides where the houses go before drawing the grass, and the model fills each part. That is the one change in the table that moved a specification by more than 0.3. The lesson generalises: give the model the decisions that are judgements (what stands in this room, who walks here) and give the code the decisions that are geometry.

Running the loop with an agent

Most of the eleven versions were written by an agent, Claude Code, working in a loop I set up and then mostly watched. One turn looked like this. Read the previous version's scores and the pictures where it lost. Write down one theory ("the model does not see enough of the map", "the sampling is adding noise", "the rules are prose, so they are advice"). Change the code with tests. Commit, so the version in Galtea records exactly what was measured. Run the 38 seeds. Compare. Write the result into the project's README, in one paragraph per version, so the next turn starts from the memory of this one.

My part was smaller and, I think, the part that mattered. I said which theory to test first. When I saw a failure the numbers missed, I asked for a specification for it, which is how four rules became seven. I set the budget: 8 by 8 grids for the bulk of the seeds, and only three 16 by 16 maps, because a room needs space to exist but a large map costs eight times a small one. And I said when to stop: when the agent could no longer name a change it was confident would move a number, the loop had done its work for now.

The agent also broke things. It once committed a version with a failing test, because a shell pipe hid the exit code. It once ran two evaluations against modules it was editing at the same time. Both were caught, one by the continuous integration check and one by the maps coming out wrong, and both became a written rule for the next turn. The loop is not a promise that every version is better. It is a promise that you will know.

What is still wrong

The corridors are stubs: a route between two doors is a shape, and shapes are the plan's job, so the next version should draw the paths the way it draws the rooms. Rooms are often empty, because the state says what the world lacks but not what a kitchen holds. Free-standing walls still split some maps in two. And rooms never share a wall, which real streets do all the time. Each of those has a number now, which is the point.

You can try the generator with your own OpenRouter key, read the code and the roadmap, or look at every map of every version, which are committed to the repository next to their scores.