varg consistent characters

Turn a scene script + actor photos into a multi-scene vertical ad where characters, outfits and location stay consistent across every cut. Tested on a 5-scene, 22-second 9:16 fitness ad with two real actors in one gym.

Generation runs on varg: the varg-ai skill (sign-in, models, local and cloud render) or the varg MCP server (https://mcp.varg.ai/mcp). This skill assumes a working VARG_API_KEY or MCP connection.

Core idea: cards, then composites

Never describe characters from scratch in every scene prompt. Build a small set of frozen reference images once, then compose every scene from them.

photo A1 ──edit──> a1Card ─┐
photo A2 ──edit──> a2Card ─┼──edit──> scene1..N ──image-to-video──> vid1..N
prompt   ──gen───> gymLoc ─┘

Upload the actor photos first (POST /v2/files, or the MCP upload tools) and use the returned URLs.

No actor photos? Design the cast with varg-character-design (archetypes, a model sheet per character on flat grey) and use each character's full-body crop as the profile card below.

1. Consistency

1.1 Character profile cards

One image-edit call per actor, from their photo:

const a1Card = Image({
  prompt: {
    text: "Full-body character reference card of the exact same woman from the photo: same face, same grey hair, same body shape. <age/build>. She wears <FIXED OUTFIT, described precisely>. Standing straight facing camera, neutral relaxed pose, arms at her sides. Plain light grey studio background, even soft lighting, head-to-toe framing, photorealistic",
    images: [A1_PHOTO_URL],
  },
  model: varg.imageModel("nano_banana_2_1"),   // edits when images are passed
  aspectRatio: "9:16",
});

Rules that make cards work:

• "exact same ... from the photo: same face, same hair, same body shape" anchors identity to the reference instead of the text description.
• Fix the outfit in the card. The card's clothes become the outfit for the whole video. Pick it so the script works (e.g. arms or legs visible if scenes point at them). Never re-describe clothes differently later.
• Full body, neutral pose, plain studio background: a clean card composites into any scene; a scene-styled card does not.

The pipeline was first built with Nano Banana Pro edit; nano_banana_2_1 is its successor and varg's default. For the hardest identity cases try gpt_image_2_5.

1.2 Location card

One text-to-image generation, no people:

const gymLoc = Image({
  prompt: "Empty modern gym interior with huge floor-to-ceiling windows... No people. ... photorealistic, 9:16 portrait framing",
  model: varg.imageModel("nano_banana_2_1"),
  aspectRatio: "9:16",
});

An empty location card lets every scene reuse the same walls, light and floor without characters from a previous scene leaking in.

1.3 Scene composites

Every scene is one edit call with all three refs:

const SAME =
  "First reference image is Woman A (<short recap of card>). Second reference image is Woman B (<short recap>). Third reference image is the gym location with huge windows. Keep both women's faces, hair, bodies and outfits exactly as in their reference cards, and the gym exactly as in the location reference.";

const scene2 = Image({
  prompt: { text: `${SAME} <scene-specific framing, poses, gestures>`, images: [a1Card, a2Card, gymLoc] },
  model: varg.imageModel("nano_banana_2_1"),
  aspectRatio: "9:16",
});

• The SAME preamble is shared verbatim across all scenes. It names each reference by position ("First reference image is...") and pins identity.
• Image edit models take several refs fine. For Video, one image = the first frame; several images switch to reference mode, which only some models support (see varg-ai).
• Scene text only describes what changes: camera framing, poses, gestures, expressions.

1.4 Cache discipline (critical)

Identical prompt + params = free cache hit, both locally and server-side on api.varg.ai. This is what makes the whole pipeline cheap:

• Keep prompts byte-identical between the images-only script, the local template and the cloud template. Copy-paste, never retype.
• The full 5-scene video re-rendered in the cloud for a few credits (the stitching fee) because every image, video and music prompt matched the earlier runs.
• Editing one scene's prompt regenerates only that scene; everything else stays cached.

1.5 Content-moderation gotcha

Body-part wording such as "rear" or "booty" in an image prompt can silently stall generation for ~10 min (provider moderation). Rephrase to neutral anatomy: "hips and thighs". The same words inside spoken dialogue in video prompts passed fine.

Fitness and swimwear scenes trip filters on words, not pictures: "girl" instead of "woman, 28", "tight", "bikini" and short prompts without a garment description get blocked on GPT Image 2.5, FLUX 3 and Grok, while a catalog-style prompt (adult age as a number, each garment named, an activity, "activewear brand catalog photo") passed on all six image models we tried. A block can be random: retry once, then rephrase, then route. Test log, safe-phrasing table and routing: varg-character-design → references/moderation.md.

2. Photos-for-approval pass (images first)

Generate and show all stills before paying for any video. A still costs a few credits; a video clip costs ten times more or more. Approve the cheap layer first.

Standalone script pattern (<project>-images.ts, run with bun):

import { Image } from "vargai/react";
import { createVarg } from "vargai/ai";

const varg = createVarg({ apiKey: process.env.VARG_API_KEY! });
// ... define a1Card, a2Card, gymLoc, scene1..sceneN exactly as in 1.x ...

async function save(name: string, el: ReturnType<typeof Image>) {
  const resolved = await el;               // await triggers generation (cached)
  await Bun.write(`output/<project>/${name}.png`, await resolved.file.data());
}

// cards + location first, then scenes (scenes reference the cards)
await Promise.all([save("a1-card", a1Card), save("a2-card", a2Card), save("location", gymLoc)]);
await Promise.all([save("scene1", scene1), /* ... */]);

Key facts:

• Image({...}) is a lazy thenable: await resolves it, .file.data() gives the bytes. Awaited results are cached, so the later video render reuses them for free.
• Run it in the background (nohup bun run <project>-images.ts > images.log 2>&1 &) and report the PID.
• Show the user: the cards + location + every scene. Iterate on individual scene prompts (only the changed one regenerates) until approved.

No bun? Generate the same stills one by one with POST /v2/image (or MCP generate_image), passing the references in files. Keep the prompts identical to the ones you will use in the render.

3. Final render: cloud (POST /v2/render)

After the stills are approved, wrap them in videos + composition and submit the whole TSX to the cloud. No bun or ffmpeg needed on the machine that submits.

3.1 Video layer: image-to-video with native audio

Dialogue is written into the video prompt and the model voices it:

const vid2 = Video({
  prompt: {
    text: `Same gym with huge windows. <scene action, static camera>.
The trainer says matter-of-factly: "<the scene's line>"
Soft gym ambient sounds.`,
    images: [scene2],                      // exactly ONE image = the first frame
  },
  model: varg.videoModel("kling_v4"),
  audio: "native",                         // generates and keeps the dialogue audio
  duration: 4,                             // integer seconds (kling_v4: 3-30)
});

• Quote dialogue lines verbatim in the prompt; add a voice direction ("clear encouraging voice", "matter-of-factly").
• Budget ~1 s of dialogue per ~2.5 words, then trim (3.3).
• kling_v3 is what this pipeline was first tested on and still works the same way (integer 3-15 s); kling_v4 is the current Kling. seedance_2_5 gives the best faces and lip movement at a higher price.
• Native audio needs audio: "native". A flat providerOptions: { varg: { generate_audio: true } } is silently dropped and you get a silent clip.

3.2 Cloud TSX rules

POST https://api.varg.ai/v2/render with {"code": "<tsx>"} (MCP: render_video with code):

• No imports, no createVarg: Render/Clip/Image/Video/Music/Captions and varg are injected as globals. Local LSP errors on this file are expected and fine.
• Must export default (<Render ...>...); no named exports.
• Same varg.*Model() ids as local.

Price it first (MCP estimate_cost with tool render and the code, or POST /v2/estimate on each clip's request) and get the user's go-ahead. Then submit and poll:

curl -s -X POST https://api.varg.ai/v2/render \
  -H "Authorization: Bearer $VARG_API_KEY" -H "Content-Type: application/json" \
  -d "{\"code\": $(jq -Rs . < template-cloud.tsx)}"
# -> {"id":"job_xxx",...}; poll every 15s:
curl -s https://api.varg.ai/v2/jobs/JOB_ID -H "Authorization: Bearer $VARG_API_KEY"
# completed -> output.outputs[0].url  (s3.varg.ai mp4)

If all prompts match a previous run, the cloud render costs a few credits (server cache) and finishes in ~2 min.

3.3 Dynamic pacing: trim dead air

Generated clips have ~0.2-0.5 s of warm-up and often trailing silence. Detect speech boundaries on the generated clips, then cut tight:

ffmpeg -i scene.mp4 -af "silencedetect=noise=-35dB:d=0.35" -f null -

<Clip cutFrom={0.2} cutTo={5.6} duration={5.4}>{vid1}</Clip>  // cut right after the line ends

This took the test video from 27 s to 22.4 s with no content loss. Keep Music duration an integer: a fractional total like 22.4 failed with a 422 on ElevenLabs music. Round down.

3.4 Centered captions in the cloud (inline SRT)

<Captions> transcribes Speech elements by itself. For dialogue that lives in a video's native audio, the tested route is two passes (newer vargai builds also accept src={vid.audio}; if the render rejects that with "Captions src must be ...", use this route):

1. Render once without captions (cloud or local).
2. Upload the audio and transcribe it on varg: POST /v2/files (upload) -> POST /v2/transcription with {"model":"whisper_large_v3","audio_url":...}. The result has word-level timestamps.
3. Build a word-level SRT; fix Whisper mishears (product names, unusual words). Captions' prompt prop can also pass context hints to Whisper.
4. Embed the SRT inline in the cloud TSX. The srt prop takes a file path, and template code runs on the render server, so write the file to /tmp at module eval:

const SRT_CONTENT = `1
00:00:00,260 --> 00:00:00,320
You
...`;
const SRT_PATH = `/tmp/project-${Date.now()}.srt`;
await Bun.write(SRT_PATH, SRT_CONTENT);
// in <Render>:
<Captions srt={SRT_PATH} style="tiktok" position="center"
          color="#ffffff" activeColor="#a3e635" wordsPerLine={3} fontSize={64} />

Re-submitting is nearly free (everything cached); only the composition re-runs. Important: if you change any clip trim, regenerate the SRT, because the timings shift.

4. QA: how to analyze what you generated

Agents can't watch video, but they can read frames and measure audio.

4.1 Visual QA: extract frames and READ them

After every render (and every images pass), pull frames and inspect them as images. Never ship a video you haven't frame-checked.

# one frame per scene, at a timestamp INSIDE that scene (mind the trims)
ffmpeg -v error -ss 3  -i out.mp4 -frames:v 1 -vf scale=270:-1 check-s1.png -y
ffmpeg -v error -ss 9  -i out.mp4 -frames:v 1 -vf scale=270:-1 check-s3.png -y
ffmpeg -v error -ss 14 -i out.mp4 -frames:v 1 -vf scale=270:-1 check-s4.png -y

• Downscale (scale=270:-1 .. 360:-1): enough to judge, cheap on context.
• Read several frames in one parallel tool call.
• No local ffmpeg? The varg MCP analyze_video tool answers questions about a rendered video.

Visual checklist per frame:

• Identity: same face/hair/body as the profile cards, in every scene.
• Outfit drift: clothes must match the cards exactly (color, cut, shoes). Drift usually means a scene prompt re-described clothing: fix the wording, not the card.
• Location drift: windows/floor/props consistent with the location card.
• Framing: matches the script (full-body vs close-up); subjects not cropped at the wrong place; both actors present when required.
• Captions: correct position (center), readable size, active-word color visible, no overlap with faces.
• Age/build drift: edit models can age or slim a person between scenes. Compare against the card and regenerate the outlier scene only.

Also verify structure numerically:

ffprobe -v error -show_entries format=duration,size -of default=noprint_wrappers=1 out.mp4

Duration must equal the sum of clip durations; a mismatch means a clip or music duration bug, not an encoding issue.

4.2 Audio QA: silencedetect as ears

silencedetect shows where speech actually lives in each generated clip:

ffmpeg -i scene.mp4 -af "silencedetect=noise=-35dB:d=0.35" -f null - 2>&1 | grep silence_

Read it per clip:

• silence at the start -> dead air / late speech start -> raise cutFrom;
• silence at the end -> line finished early -> lower cutTo;
• no silence in an N-second clip -> speech fills it; if the line is long, the model may have rushed or truncated it: listen-check via transcription;
• silence in the middle of a one-line clip -> possible stutter or split delivery.

Then confirm the words: transcribe the final mix (3.4 step 2) and diff the transcript against the script. This catches mispronunciations by the video model and mishears by Whisper. Fix mishears in the SRT; regenerate the clip only if the model actually said the wrong thing.

4.3 When to regenerate vs when to trim

Symptom	Fix
Dead air, late start, early finish	Trim with cutFrom/cutTo: free
Wrong/garbled spoken line	Regenerate that Video
Identity/outfit drift in one scene	Regenerate that scene Image, then its Video
Caption timing off after re-trim	Regenerate the SRT only (one transcription)

5. No script? Analyze the reference video first

When the user sends a reference video instead of a script, reverse-engineer the same A/L/Scene structure this skill consumes. Do this before touching any generation. (media-download fetches a reference by link.)

5.1 Extract structure

ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1 ref.mp4

# scene cut candidates (hard cuts)
ffmpeg -i ref.mp4 -vf "select='gt(scene,0.35)',showinfo" -f null - 2>&1 | grep showinfo | grep pts_time

# speech map
ffmpeg -i ref.mp4 -af "silencedetect=noise=-35dB:d=0.35" -f null - 2>&1 | grep silence_

Scene-cut timestamps + speech gaps together give the beat structure: how many scenes, how long each, where dialogue sits.

5.2 Sample frames per scene

Extract 1-2 frames inside each detected scene and read them (parallel call):

ffmpeg -v error -ss <t> -i ref.mp4 -frames:v 1 -vf scale=360:-1 ref-s1.png -y

From the frames, document for each scene:

• Actors: who is on screen, rough age/build/hair -> becomes A1, A2...
• Wardrobe: what each actor wears (this defines the profile-card outfits).
• Location: environment, light, key props -> becomes L1 (L2...).
• Framing: full-body / medium-wide / close-up; front or back view; camera static or moving; who is closer to camera.
• Action + gesture: what each actor does (squats, points at X, thumbs-up).

5.3 Get the dialogue

Transcribe the reference audio (same varg call as 3.4). Map each line to its scene via the timestamps. If there's no speech, note the music mood and on-screen text instead.

5.4 Write the script, then confirm

Assemble the standard format and show it to the user for approval before generating anything:

Actors:
- A1: <description from frames>  - C1: <wardrobe>
- A2: <description>
Location:
- L1: <description>
Scene 1 (Ns): <framing>. <action>. A2 says: "<line>". L1
Scene 2 ...

From here the pipeline is identical: cards (1.1) -> location (1.2) -> composites (1.3) -> approval pass (2) -> videos + cloud render (3). Use the reference's measured scene durations as the initial duration values, and its framing notes verbatim in scene prompts.

Cost reference (5 scenes, ~22 s at 9:16)

Rough credits from the varg catalog (October 2026). Always quote the user the exact figure from POST /v2/estimate / estimate_cost, in credits.

Step	Credits (~)
8 images (2 cards + location + 5 scenes), nano_banana_2_1	~65
5 Kling clips with native audio, 4-6 s at 720p	~400-500
Music bed (mureka_9_5) + one transcription	~30
Cloud re-render with everything cached	a few (stitching fee)
Iterating one scene image / one scene video	~8 / ~80-100

Related skills

varg-ai (sign-in, models, render formats), varg-character-keynote (one speaker on a stage), varg-podcast-video (two seated characters in dialogue), varg-previz (plan many shots and camera moves first), media-download (fetch a reference video), video-skills (find a skill for another format).