Test Image and Video Models on Colab GPUs: One VM, Mix and Match

This is the general method behind the narrated short blueprint. That article built one film with one model pair on one GPU. This one is about choosing the pair and the GPU.

What One VM Looks Like

ComfyUI runs as an HTTP server on the VM. Each stage is a saved API workflow JSON, and a small client posts the right workflow to 127.0.0.1:8188. The image workflow and the video workflow live in the same server and share the same disk.

Colab VM (one runtime)
├── ComfyUI server :8188
│   ├── workflows/image.api.json   keyframe model
│   └── workflows/video.api.json   image-to-video model
├── models/                        every weight file for both stages
├── story.py                       reads the storyboard, calls both workflows
└── /content/story/<name>/         frames/  clips/  (persist across reruns)

ComfyUI loads a model when a job needs it and unloads it, to system RAM first, when another job needs the memory. So the sum of the models does not have to fit in VRAM at once. The sum has to fit on disk, and the largest single stage has to fit in VRAM or offload cleanly to RAM.

nvidia-smi --query-gpu=name,memory.total,memory.used,utilization.gpu --format=csv
df -h /content

Measured Runs

MiniMax H3

H3's weights are about 38.6 GB in the template-matched set, as the H3 on ComfyUI guide describes. That guide ran on a 96 GB G4. The A100 run shows the same stack fits on an 80 GB card with room left for a second model.

Wan 2.1 FLF2V

The clip stage peaks at about 52.6 GB: that peak exceeds a 40 GB card, and the stack has not been run on one. The same request to rent an A100 can hand you either shape, so check the card before installing anything.

This row is the current film's stack: the same storyboard and keyframes animated with Wan instead of H3, at a smaller canvas and a shorter effective clip. Both rows ran on an A100 80GB in separate sessions; the workloads differ (H3: 124 frames at 1344×768, 8 steps; Wan: 81 frames at 1280×720, 4 steps), so compare per-frame cost with care.

The Test Matrix

The point is to change one thing at a time. Hold the storyboard, seeds, resolution, frame count and turbo setting constant.

GPUs

Colab's runtime types on a paid plan typically include T4, L4, A100 and G4, with an H100 on higher tiers. Check what your plan offers in the runtime dialog. Since late September 2026, Google AI Pro (priority GPUs, 200 compute units a month) and Google AI Ultra (1,000–2,000 units, background execution, premium GPUs) include Colab's paid benefits — if you already subscribe for Gemini, you may already have the faster cards without buying a separate Colab plan. Claim it from Colab's paid products page (upgrade to Google One), then check it under Settings → Subscription; paid plans only — no trials, no storage-only Google One. See Working in Colab for what the plans changed.

Image models

Video models

A model joins the matrix only if it has a ComfyUI API workflow and a license that allows your use. H3's license is not the permissive kind, so read it before building on it. For a permissive (Apache-2.0) alternative, Wan 2.1 FLF2V + lightx2v is the model used in the revised Last Lantern cut.

What to Record Per Run

Write one row per (GPU, image model, video model) combination. The columns are cheap to collect and answer the question "which is the best fit":

A quality rubric

Speed means nothing if the clips are bad. Score each finished video from 1 to 5 on the same four questions, ideally with someone who hasn't seen the settings:

1. Identity: does the main subject stay the same from clip to clip?
2. Flow: do the clips connect as one take, or dissolve and cut?
3. Motion: is the movement plausible, with no melting or jitter?
4. Prompt match: does each clip do what its motion brief said?

Choosing a Combination

Use these rules, then confirm them with the matrix:

• Start from the video model. It is the largest stage, so it sets the minimum GPU. Pick the card, then fit an image model around it.
• Disk first, VRAM second. Every weight for both stages sits on disk the whole session. Colab disk runs roughly 100 to 230 GB depending on the machine, and the video stack alone can use 40 GB or more.
• Don't pay for headroom you don't use. If the video stage peaks well under the card's VRAM, a cheaper card may do the same job, only slower. Run the clip stage on both and compare seconds per clip against what the card costs you.
• Keep one VM only while both stages run. Once keyframes are approved, the image model is dead weight. Download the frames, then decide whether the video stage needs a bigger runtime.

Rules That Held Across the Run

• Look at every keyframe before you generate video. A clip animates whatever the frame shows, including mistakes. The two-frame structure also means a changed keyframe invalidates the clips on both sides of it.
• Keep work on the VM resumable. Frames and clips persist under /content/story/<name>/, and a rerun skips what exists. A dropped local session does not stop the VM.
• Isolate extra tools in virtual environments. The VM's Python was 3.13, and several audio and speech packages don't support it. A uv venv per tool keeps ComfyUI's environment intact.
• Match the PyTorch build to the card. A newer card can need a newer PyTorch build, and a mismatched torchvision can fail with a CUDA-version error even when torch itself loads.

Next Experiments

1. Run the same storyboard on an L4 with a smaller image model, to get the first real row for a cheaper card.
2. Run the video stage on a G4 and on the A100, same stack, to compare speed against the size of the card.
3. Swap in a second video model in the same slot and score both with the rubric.
4. Fill in the matrix, then publish the winners per GPU.

What's Next

• Narrated anime short pipeline: the full build this method was extracted from
• MiniMax H3 on ComfyUI: the video stack in detail
• Qwen-Image on ComfyUI: an image stack for smaller GPUs
• Working in Colab: runtimes, compute units and the Colab CLI
• Hugging Face models in Colab: budgeting disk and VRAM before you download