Everything we got wrong, and what each mistake cost

The catalogue · nineteen mistakes, days one to five

Every project like this publishes its successes. The failures are more useful, and they are cheaper to read than to repeat — so here is every significant thing we got wrong, sorted by the kind of mistake it was, with what it cost.

At the end is the part that surprised me most: a count of what actually caught each one. Careful thinking caught none of them.

Class one

Believing something without measuring it

The most common class, and the most embarrassing, because every one of these was cheap to check and we wrote it into a plan instead.

What we believed	What was true	Cost
Keeping the text encoder in system RAM is mandatory on a 32 GB card	It made the same shot 24.3% slower. A 6-second job on the graphics card became a 110-second job on the processor, and the constraint it solved was not real	one render, and a written-down certainty
Handing the GPU between two programs is expensive enough to need a custom scheduler	About 30 seconds, under 5% of one shot	none — and it deleted three planned components
Packing several angles into one long render is cheaper	Cost per frame rises with length, so it is more expensive and you lose the ability to re-roll one angle	an assumption in a plan
0.65 is the obedience setting at which two cameras describe one room	It holds one camera. The sweep that produced it used one camera	a production default, superseded the next day
Shots sharing a single render get continuity “for free”	Backwards. A shared render shares the noise — it wins actor position and loses identity, wardrobe, geometry, appearance and lighting	a documented rule, inverted
Anti-aliasing the depth maps will sharpen the result	Moved the finished detail from 4.459 to 4.473 for four times the drawing time. Noise	an hour, spent deliberately to find out

Class two

The answer was in a document we had not opened

This class is worse than the first, because there was nothing to discover. Someone had already written it down.

• We never read the prompting manual. The video model ships a strict field-based format, and identity is bound by declaring a subject and marking it preserved — not by writing “keep her face exactly”, which is what I had been writing. Everything achieved in the first two days was achieved despite the prompts. Gabe asked whether I had checked the official docs. I had not.
• We were using one reference slot out of nine. The model’s own guide states, in as many words, that one subject may be defined by several reference pictures.
• The model has a closed camera vocabulary, and we never used it. Which means a whole day spent concluding “nothing carries the camera” had tested the wrong thing.
• The reference picture had been contradicting our own specification since the day it was made. The hall kept rendering as a domed theatre. Its own embedded metadata asked for “three sweeping tiers of crew stations” under a ceiling “six storeys high” — while the specification said two straight banks under a flat 4.5 m ceiling. We spent two days rewriting prompts against an image that was arguing with us.

Four of the six experiments chasing one bug were wordings. All four failed, because the driver was the picture.

Class three

Silent failures that look exactly like results

The dangerous class. In each of these the machine produced a well-formed, plausible output that answered a question nobody had asked.

The fault	What it looked like	Cost
A nested key instead of a flat dotted one. The reference input is declared as a flat dotted name and is optional, so a nested object matches nothing and the render proceeds as pure text-to-video	A bald, bearded man who was not our actor — consistently enough to look deliberate. I proposed three prompt-level explanations. All three were wrong	15 renders, plus two confident false conclusions
The image loader caches on the filename. A depth map regenerated and re-uploaded under its own name is never re-read	A byte-identical plate from a map that had changed by 7,000 bytes. A well-formed render of the right prompt at the right seed — and the answer to the previous question	two false conclusions, both published
Sixteen-bit depth maps are converted by clipping, not scaling	A ramp of 1,024 distinct values arrived as five values followed by pure white. Not degraded — actively wrong, and confidently so	a promising route in two open questions, closed
Three camera arms inherited the first camera’s depth range	One arm’s map held only a wall, a floor and one desk. The model filled the void with a staircase with handrails that exists nowhere in the set	one experiment’s arms were never equally controlled

Class four

The instrument was lying

Once you start measuring, the measurements become the thing that can be wrong. On one day we built six new instruments and five of them gave wrong answers. One had already published a false failure before it was caught.

• A travel measure that summed sub-pixel drift to zero, reporting a moving camera as stationary.
• A detail metric that ranked the winning plate worst, until resolution was normalised.
• A galaxy locator that kept finding desk lamps. It produced a tidy, entirely false conclusion that the galaxy and the actor had swapped places.
• A costume measure that scored a known-good control as a failure.
• A camera-travel measure that saturates at ±160 pixels and, once saturated, wraps onto the repeating architecture of the set and returns a confident wrong number — sometimes with the wrong sign. It reported a false failure for a tracking shot and understated a camera arc by a factor of ten.

Class five

Comparing things that cannot be compared

Three mistakes of method, and the first is the reason this project now has a rule about what counts as evidence.

• I concluded “fewer references give a better likeness” by comparing two renders with different prompts. Likeness varies more between prompts than between reference configurations, so the comparison measured nothing at all. It produced a confident, entirely false finding — and it is why only same-seed, same-prompt arms are evidence is now the first rule of the project.
• Four of five cameras obeyed a broken convention by coincidence. Facing forward, screen-left simply is port — so a broken rule returned the right answer four times and the wrong answer once, and that one wrong answer looked like a rendering defect rather than a specification defect. Coincidental agreement is the most expensive kind of passing test.
• I published an explanation before running the experiment that settled it — on the page Gabe uses to choose plates. It was wrong. A confident wrong answer in a working document is worse than no answer.

Class six

The record itself

A project that keeps its own notes eventually has to check them, and ours had grown past the point where reading it was enough.

Fault	Detail
A research document we cited three times has never existed	The index carries a full row for it marked current. An experiment says a decision was taken “after research ruled out the alternatives”. A later document cites it again. There is no file and there never was. The conclusions survive; the reasoning is gone
The render ledger rewrote itself from 385 rows to 122	Every still silently dropped, no error, exit status zero — an image library import sat inside the same error handler as “this file has no embedded workflow”, so a missing library looked identical to an empty file. Caught only because a printed count disagreed with another document
Every rebuild erased the human verdict field	The project’s first properly graded sequence would have survived exactly until the next time anyone ran the tool
The most-used asset in the production was unprotected	The face photograph in 49 renders — more than twice anything else — existed only inside the renderer’s scratch folder

---

The part I did not expect

What actually caught each one

I went back through all nineteen and asked a single question of each: what was the thing that finally revealed it? The distribution is lopsided, and the empty bar is the interesting one.

If you want the version with the pictures and the numbers, it is in the five days. If you have not read how a shot is actually made, that comes first and takes about four minutes.

Every entry above is traceable to a numbered experiment, decision or session in the project’s own record. Where a claim was withdrawn, the withdrawal is recorded in the same place as the claim. Nineteen is the count of mistakes significant enough to change a decision or waste machine time; smaller ones are not listed.