A blank sky is an instruction to paint a wall

Two frames of the same street: on the left plain plastic-looking surfaces and a blank kiosk, on the right the same view with newspapers in the kiosk, carved stone surrounds and paving joints

Session S-034b · Experiments E-071 and E-072 · 16 August 2026

Gabe, on the frames I had just called a success: “these last shots are NOT photorealistic — you’re aware of it yes? they are correct, but really not well done — they look like a 90’s 3d engine’s render.”

He was right, and my write-up had not mentioned it. The cause turned out to be a sentence I had written that same morning and failed to follow one step further: a picture you hand the machine teaches it everything about itself, not only its shape.

Wrong guesses first

2

Surface variation, before

3.13

After

4.65

Steps asked for

30

Steps actually run

21

A street scene with correct geometry but plastic surfaces: a blank green kiosk with no contents, plain slab buildings, featureless shopfronts and smooth grey pavement
Correct, and plasticA blank kiosk. Slab buildings. Smooth everything. This is what Gabe was looking at.
The same street from the same camera, now with newspapers visible inside the kiosk, carved stone window surrounds, wrought-iron balcony grilles, paving joints and drain covers
Same camera, seed, words and drawingNewspapers in the kiosk. Carved surrounds, iron grilles, paving joints, a drain. The only thing changed is the surfaces of the little render it was started from.
One variable. The machine, the description, the random seed, the distance drawing and every setting are identical between these two frames.

Section 1

It was never the model

The obvious suspicion, when pictures come out looking like a video game, is that you have reached the limit of the machine making them. That was easy to rule out and the answer was uncomfortable. The same model, the same description, the same random seed, started from noise instead, produces wet granite setts, newspapers clipped to a rack and reflections in plate glass. Started from my little render of the set, it produces plastic.

The day before, I had found that widening the tonal range of that starting render changed how much of it survived into the photograph, and I wrote the reason down: the picture you start from teaches, and it teaches everything about itself. I then applied that to brightness and stopped.

Every surface in that render was one perfectly uniform, noiseless value. A wall was a single grey. A pavement was a single grey. That is not an absence of information about texture. It is a clear statement that this material has no texture — and the machine believed it, faithfully, everywhere.

Section 2

Photograph the maquette, do not draw it

The fix follows directly. The starting render stops being a technical drawing and becomes a photograph of a physical model: three scales of mottling laid over every face so nothing is a single value, a fine grain over the whole frame, a gentle darkening at the corners, and a touch of colour fringing at the edges.

All four were already written down

Grain, vignette, fringing and surface variation are things the description has been asking the final photograph for all along. The render had none of them, and it was sitting underneath the description contradicting it. The anchor has to be the kind of picture you are asking for, not a diagram of it.

Measured on the frame, local variation across small patches went from 3.13 to 4.65. Measured by eye, the kiosk that had been an empty green box has newspapers in it.

One more thing had to move with it, and it is a trap worth naming. Keeping 30% of the starting picture means the machine skips the first 30% of its own process — so a request for 30 steps at that setting actually runs 21. Texture is what the late steps are for, and I had been quietly buying nine fewer of them. It runs 50 now.

Section 3

“The background in the piazza is a wall instead of sky”

That was another of Gabe’s notes, it was the hardest one, and my first fix for it made it worse.

A convincing Italian street where the far end of the view, which should be open sky above a square, is filled by a flat marbled white plaster wall
Where the sky should beA rendered plaster wall, closing off the square. Everything else in the frame is right.
The same street with the far end now open to a soft overcast sky, with the street continuing away past the buildings
With cloud, a gradient and a cool cast in the anchor’s skyThe wall is gone and the street opens.
The only difference between these frames is what the sky looks like in the little render underneath them.

Guess one: the mottling. Blotches on flat pale grey is exactly what weathered stone looks like, so I took the mottling out of the sky. It fixed one camera completely and did nothing at all for the other, which should have told me immediately that it was not the mechanism.

Guess two: hold it harder to the drawing. Reasonable, and it exposed a real limitation on the way to being refuted. In the grey distance drawing, sky is the value zero — and so is every real surface at or beyond the far limit of that drawing. They are literally the same number. The channel we use to say where things are cannot distinguish “there is nothing here” from “there is a wall at the far limit”, and there is currently no second channel that can. It is a genuine hole and it is now written down as an open question. It was also not the cause: holding the drawing harder produced a beautiful frame with the wall still in it.

The cause was that I had made the sky featureless, which is not the same as neutral.

A large, perfectly uniform, hard-edged pale field is what a rendered plaster wall looks like — and the machine had just been told, by the rest of the anchor, that this is a photograph of a physical model. Taking the mottling out was half right; the other half made it worse. The measurement is unarguable: the anchor’s sky was given as a blue-grey (204, 208, 213) and handed back as (203, 203, 201). It desaturated the sky into stone.

An overcast sky is never featureless. It has very large, very soft cloud; it is much brighter overhead than at the horizon; and it is cool. The anchor’s sky now has all three — a vertical ramp from 246 at the top to 174 lower down, a cast fifteen points bluer than it is red — and it is exempt from the grain, the vignette and the fringing, because a sky is not a surface of the maquette. The wall is gone.

Section 4

And then it invented a seafront

With the sky fixed, the far end of the second camera came back as palm trees and a promenade. Same class of fault, one step along. That camera had inherited its far limit — 110 m — from the camera pointing the other way. Turned round and moved 52 m up the street, it is looking at a frontage that runs from 71 to 136 m away. Its entire terminus was clamped to black, which is the same value as sky, so the machine was handed a void where the end of the street should be and did what it is for: it filled it.

A camera’s far limit has to be computed from that camera’s own view, never inherited from a neighbour. Raising it from 110 to 140 m costs a little of the drawing’s precision — the smallest step it can describe goes from 3.4 m to 4.3 m — and that matters far less than it used to, because the anchor now carries everything below that threshold anyway. The two halves of this session fit together.

Section 5

Where the dials sit now

KnobSettingWhy
What the picture starts fromA flat-shaded render of the setThe place, instead of a blank page
The anchor’s surfacesMottled and grained, sky cloudedA photograph, not a diagram — this post
How much of it to keep0.700.50 gives a cartoon; 0.80 gives a different street
How hard to hold the drawing0.850.70 loses an eighteen-pier arcade from a picture start
Steps50Because 0.70 × 30 actually runs 21
Every one of these is one render per arm on one camera. They are a working point, not a measurement.

The general form, which is worth more than the setting.

Anything you hand a generative model as a starting point is a full description, including the parts you did not think you were describing. A uniform surface says untextured. A blank sky says wall. A clean edge says rendered. There is no such thing as leaving a property unspecified in a picture — you can only specify it accidentally. The session this came out of is here, and the next thing to find out is whether any of it survives a camera that moves.

Sources: the two experiment write-ups of 16 August 2026 and the three decisions that came from them. Every figure is one render per arm at 2048×1152, single-variable against the frame beside it, and the files are in out/e071_photoreal and out/e072_notes. The surface-variation figure is the mean local spread over 8×8 patches; the sky colours are the mean of the region above the roofline in the anchor and in the returned frame.