Day one: everything we believed was wrong

Day one · 8 August 2026 · sessions S-001 to S-010

We began with a clear question, a sensible plan, and four confident beliefs. By the end of the day three of the four were dead — not argued away, but measured and buried. One of them was mine, and it was embarrassing.

In between, the pipeline produced its first real shot on the first attempt, which was the only thing all day that went as expected.

Where we started

A 33-billion-parameter model and a card that cannot hold it

The video model is MiniMax H3, released five days before we started. It is unusual: 33 billion parameters, and it generates picture and 32 kHz stereo sound in a single pass rather than dubbing audio on afterwards. It is also guidance-distilled, which in plain terms means there is no negative prompt — you cannot tell it what you don’t want.

The machine is a single desktop: an RTX 5090 with 32 GB of video memory and 96 GB of system RAM. The arithmetic that shaped the whole first day: the model’s transformer needs 21 GB, and its text encoder — which is itself a 32-billion-parameter language model — needs another 16 GB. They do not fit at the same time.

So the day’s brief was straightforward: find every way to make this run leaner. I ran three parallel research streams through the model card, the renderer’s issue tracker, the papers and the practitioner reports, and produced a ranked list of levers. It was a good list. It was also answering the wrong question, and Gabe corrected it within the hour.

Wrong belief #1

This was never a “make it smaller” problem

What I had missed was what else lives on the card. The same GPU runs a 27-billion-parameter language model — Qwen 3.6, under an agent harness called Hermes — whose job is to take a script, turn it into per-shot instructions, and drive the renderer unattended overnight. The point was never a lean renderer. The point was two large models taking turns on one card without tripping over each other.

That reframing changed the answer completely, and the useful finding fell out of it immediately: batch, do not interleave. Swapping models between every shot costs about a quarter of an overnight run. But planning is separable from rendering — so plan every shot, unload the planner once, render the entire queue, then reload the planner to review. Two swaps a night instead of a hundred.

Build the gap, not the overlap. Researching the agent harness properly showed it already had the scheduler, the batch runner, the subagents and a local API. The one thing it did not have — and that existed nowhere else — was arbitration of a single GPU between two competing programs. That gap is small. Everything around it was already built, and we nearly rebuilt all of it.

The one thing that worked

Eight seconds over Manhattan, first attempt

Gabe picked the test: a photorealistic shot from inside an F-35 cockpit over Manhattan, eight seconds. Three candidate stills in 32.8 seconds, then the chosen frame drove a video render with generated sound in 608 seconds. No failures, no retries.

Two failure modes surfaced while writing the prompt, and both are the same mistake in different clothes. Describing the camera as being behind the pilot put the camera outside the aircraft. Naming our own aircraft’s exterior produced a second aircraft, flying ahead of us. The model does not know which object is the vessel you are inside; name a thing and it will happily put that thing in front of the lens. The fix was a tight over-the-shoulder frame with no airframe described at all.

Wrong belief #2

My best idea made everything 24% slower

Here is the one that was mine. Given that the transformer and the text encoder cannot both fit in memory, the obvious move is to keep the encoder in system RAM — there is 96 GB of it — and leave the card free for the part doing the heavy work. I had written this up as mandatory, not an optimisation. It was the single most confident claim in the day’s research.

Then we measured it.

The second measurement killed a matching pile of complexity. We had assumed handing the GPU between the two programs would be expensive enough to need careful engineering — a custom lease, a scheduler, a merged graph. In fact release-and-reload costs about 30 seconds, under 5% of a single shot. Below that threshold the supervisor is allowed to be dumb and coarse. A custom node, a mega-graph and a serialization layer all went in the bin, unwritten.

Wrong belief #3

We had not read the manual

Late in the day Gabe asked a plain question: had I checked the official prompting documentation? I had not. He was right to ask, and the answer reframed everything that came after.

The model does not want prose. It ships a strict field-based schema — separate fields for the integrated description, the overall soundscape, the non-diegetic music; and for reference-driven shots, explicit subject definitions with retention markers saying how faithfully each reference must be honoured. Identity is bound by declaring a subject and marking it preserved, not by writing “keep her face exactly”, which is precisely what I had been writing. Camera moves are a closed vocabulary of type, amplitude and speed, not free description.

And the detail that explains a great deal of public disappointment with this model: the hosted commercial version runs an unreleased preprocessing layer that silently rewrites your prose into this schema for you. Run the open weights locally and there is no such layer. You must author the structure yourself, or you are using the model with one hand tied.

Everything we had achieved so far, we had achieved despite the prompts. It was a floor, not a ceiling.

So we ran it properly: same references, same settings, same seeds, changing only the prompt format. Identity held either way — but environment fidelity, eyeline accuracy, spoken performance and camera execution all improved markedly, at no extra render time. The retention markers do real work: asking for a background to be partially preserved produced a soft, plausible background where fully preserved produced a literal copy.

And one more thing

Nine slots, and we were using one

The last finding of the day was the cheapest and the most annoying. The model accepts up to nine reference images. Its own guide states, in as many words, that one subject may be defined by multiple reference assets — several photographs of the same face, all bound to the same character.

We had been sending one.

I also set an honest expectation, because it would have been easy not to. The one clean published measurement puts multi-reference identity gain at roughly +0.8 percentage points. That is real, and it is free, and it is not a step change. The step change would be training a likeness — which is a different and much larger piece of work.

One negative result, recorded because it was expensive

We tested an appealing idea: composite the real face into a still first, then animate the still. It was worse — weaker likeness, synthetic-looking skin. The cause is structural rather than incidental: two generative hops instead of one, and freezing the first hop’s output as the first frame makes its drift canonical. It also pins the camera, since likeness survives best at the source photograph’s head angle. Abandoned mid-test, by agreement, and written down so nobody tries it again.

---

What day one settled

The shape of the whole project, in one day

Looking back from later in the week, the pattern that runs through everything was already fully formed on the first day. Every problem we had was upstream of where we were looking for it. The speed problem was a scheduling problem. The identity problem was a prompt-format problem. The prompt-format problem was a reading problem.

And the correction always arrived the same way — a measurement, or a primary source, contradicting something we were sure of. Not once, all week, did an argument settle anything.

Sources: docs/01_SESSION_LOG.md S-001 to S-010 · experiments E-001 to E-007 · research R-001 to R-009 · decisions D-001 to D-025. Render timings from the shot ledger. Software: ComfyUI 0.31.1, MiniMax H3, Z-Image Turbo, Ollama with Qwen 3.6 27B, Hermes Agent 0.20.0.