A ready-to-run ComfyUI workflow for multi-shot talking-character video: one recurring character across completely different locations, with the same face and the same voice — audio generated together with the picture, not dubbed.
The showcase video is exactly what the workflow does: five shots, five locations, one woman, one voice. Nothing carries her between shots except a single identity sentence repeated byte-identically in every prompt, plus the cross-shot memory bank this graph wires up for you.
Before you dive in — a word on expectations. This is a community-built, bleeding-edge pipeline: a 22B audio+video model with cross-shot memory, running on consumer hardware. It is powerful, but it is not one-click — your first clean render will likely take some tuning to YOUR machine (VRAM, system RAM and pagefile, and which model build). No two rigs behave identically. INSTRUCTIONS.md in the zip covers install, per-VRAM settings, and every failure mode reported so far. If you get stuck, comment here or open a GitHub issue — I answer, I troubleshoot, and most of the fixes in this pack exist because someone reported something. You will not be left hanging.
Quick fixes — read this first
Nearly every problem reported with this pack is one of these.
Renders crawl; VRAM pinned at 100%
Why: Withoutfp8_scaled_mmthe DiT runs in bf16 — ~40 GB staged. On a 32 GB card that streams over PCIe every step.
Fix: Turnfp8_scaled_mmON (with a bf16 or INT8 checkpoint) andsequential_offloadOFF — ~22 GB resident, native fp8 matmul on RTX 40/50-series. It is off by default and is the single biggest speed setting in this pack.fp8_scaled_mmerrors — “needs the bf16 checkpoint as its source”
Why: The toggle downcasts the attention/FF linears itself, so it must start from bf16. An fp8 FILE would load every tensor as fp8 — norms, tables and adalns included — and crash the denoise pipeline.
Fix: Pointmodel_fileat the bf16 (or INT8) build. Do not pair the fp8 file with this toggle.The fp8 file didn't speed anything up
Why: With the toggles off, an fp8 checkpoint upcasts to bf16 at load. It saves download size, not memory or time.
Fix: Use bf16 or INT8 +fp8_scaled_mmfor the actual win.The INT8 file is slow
Why: You ran it withoutfp8_scaled_mm. INT8 reconstructs to bf16 at load, so on its own it is the bf16 path plus ~40 s of reconstruction.
Fix: Turnfp8_scaled_mmON. INT8 then behaves exactly like bf16 as a source — same ~22 GB resident — at roughly 60% of the download.Lip sync drifts apart ~10 s into every shot
Why: The wrapper's video RoPE clock was hardcoded to 24 fps while audio RoPE runs in true seconds — a 25 fps render drifts ~4%/s. Looks like a model limit; it is not.
Fix: Apply the multishot patch (Bug fix #0). No checkpoint can fix it. With the patch, 60–105 s multishot masters hold sync.memory_size=0every shot; a new face each shot
Why: Withenable_audio_memoryoff, the video memory-bank save was gated on the audio latent, so the bank never filled.
Fix: Fixed in the patch (Bug fix #1). Consolememory_size=should climb 0,1,2,… up to your cap.Quality degrades over a long run — waxy skin, smearing by the late shots
Why: The memory-bank trim was a no-op whenmemory_max_size <= num_fix_frames, so the bank grew unbounded.
Fix: Fixed in the patch (Bug fix #1b).memory_size=now freezes at your cap.A GGUF errors about VAEs
Why: A GGUF is DiT-only.
Fix: Keep a full bf16 checkpoint incheckpoint_path— it supplies the VAEs, vocoder and connectors.Burned-in subtitles or captions
Why: The DMD pipeline has no CFG, so a plain negative does nothing.
Fix: Usenegative_prompt_video/negative_scale_video(~0.5). Above ~0.8 it locks every shot to shot 1's composition.The voice comes out British
Why: LTX drifts British unless told otherwise.
Fix: Name it in the positive: “in a casual American accent”.
What's in the zip
The workflow (11 nodes, grouped and labeled: Prompt Source → Enhance & Encode → Models & Reference → Generate → Finish & Save)
Built-in note blocks explaining which widgets to set, which to leave alone, and what to change on bigger cards
A 2-shot example prompt file demonstrating the identity-sentence rule
A README with setup, prompt-writing guidance, and performance notes
What you need (not included)
Node pack: RealRebelAI's
ComfyUI_JoyAI_Echo_GGUF_Nodesplus my multishot patch (v1.2 or newer required)Checkpoint: the echoVid+ltxAud surgical merge — bf16 (43 GB) or the INT8 ConvRot build (27 GB)
Text encoder: any Gemma-3-12B-it, single-file safetensors or GGUF
Setup is two things
Pick your Gemma in the Model Loader's
gemma_filedropdown.Drop
example_multishot.jsonintoComfyUI/input/joyecho_prompts/and pick it in the Prompt Source node. Queue.
Everything else I've published
JoyAI-Echo multishot patch — nodes, releases, issues
LTX-2.3 Multishot Lite — the core-node workflow
Support this work
Everything I publish — workflows, merges, quants, patches — is free and stays free. The models are non-commercial by license and I have no plans to change that.
If something here saved you an evening:
GitHub Sponsors — recurring, no platform cut
Ko-fi — one-off, no account needed
Liberapay — recurring, open-source and non-profit
Bug reports are worth as much as money. Most of the fixes in these packages exist because somebody posted a log.
Description
**Nodes, workflow, releases and issue tracker:** https://github.com/jlucasmcrell/joyai-echo-multishot-patch
**All models on Hugging Face:**
https://huggingface.co/joeygambino
**v1.5 — the lip-sync release. The ~10-second drift is dead.**
**Lip-sync drift past ~10 seconds: FIXED.** Since the first release, every shot's lip sync would quietly drift "off by a hair" starting around the 10-second mark — mouths slightly ahead of the audio, in every shot, on every model, at 25fps. Root cause: the pipeline's video positional clock was hardcoded to 24fps while renders played at 25, a 4% timing skew that accumulated ~40ms per second until it crossed visibility right around 10s. The clock now follows your actual `video_fps`. Verified: 361-frame shots (15 seconds) with frame-perfect sync, end to end, at both 24 and 25fps. The old workarounds — short spoken lines, keeping shots at 241 frames — are obsolete. Write long shots. Let them talk.
**Hires refine v2 — window-coherent noise.** The hires pass previously drew independent noise per processing window, which caused a subtle texture "switch" about once a second. All windows of a shot now share one noise field per denoising event: texture identity holds across the whole shot. The `hires_denoise` dropdown is now four honest modes: `subtle` / `medium` / `strong` (real synthesized detail — pores, hair, weave — at increasing strength, with a small per-frame texture-shimmer tradeoff that grows with depth), and new **`spatial`** — the LTX latent upsampler applied to the shot's own latents: fully deterministic, zero temporal shimmer, fixed 1.5x. Spatial needs canvas heights whose /32 is even (768, not 736) or the model smears one edge — that rule is in the tooltip.
**One thing to understand before touching the hires widgets: `hires_factor` is a routing switch, not a quality slider.** At 1.0 (the default and the recommendation), your master is built from the base shots with the deterministic upscale — zero detail shimmer. Above 1.0, the selected `hires_denoise` pass runs in-render and the master is built from *its* output instead. Raise it only when you deliberately want the refine's synthesized detail (with its slight per-frame texture shimmer) or the `spatial` mode (even /32 canvas dims only — 768-height, not 736). And always judge results from the `*_MASTER.mp4` file, never the in-graph preview (now saved with a `PREVIEW` prefix) — that preview's re-encode shows artifacts your master doesn't have. The full routing table is in INSTRUCTIONS.md.
**AutoFinish, smarter and cleaner.** The background master-builder now detects hires runs and assembles the master from your `shot_hires` files directly (no more double-upscaling from base). For base-res runs the default upscale is now **bicubic + contrast-adaptive sharpening** — deterministic, seconds per shot, CPU-only, and it never reshuffles fine detail the way neural per-frame upscalers do (we tested them all, including diffusion restoration models; the boring answer won a fair fight). RTX upscaling remains one click away as `rtx (legacy)` on the AutoFinish node. The master's glitch pass also re-encodes with the same clean settings as the per-shot files now, so finished masters no longer pick up B-frame quality pumping.
**Loader fix.** Quantized loads (`fp8_scaled_mm` / `fp8_transformer`) now cast stray float32 tensors correctly — fixes the `mat1 and mat2 must have the same dtype` crash some checkpoints hit at load.
---
**Also included since 1.3.1** (the v1.4 changes, which never got their own Civitai version): multiple LoRAs via a chainable LoRA Stack node with per-slot strength dropdowns; a visible, documented Reference Image node; `INSTRUCTIONS.md` in the zip — a complete manual covering install, first render, prompt-writing rules, per-VRAM settings, and a troubleshooting table for every failure mode reported so far; audio memory defaults ON (cross-shot voice consistency) with the tradeoff documented in its tooltip; `head_trim_frames` defaults to 14 (trims the unstable first half-second of each shot, where identity morph concentrates).
As always: the zip is the complete pack — nodes, workflow, example prompts, manual. Copy over your existing install, restart ComfyUI, press R.FAQ
Comments (7)
Hello,
It seems that your code and instructions are mixed up—some nodes are actually included in your ZIP file but are listed as not being there, while others are different versions of the original nodes (RealRebelAI's ComfyUI_JoyAI_Echo_GGUF_Nodes). Additionally, regardless of the combinations with checkpoints or diffusion models, there are quantization errors. Your idea seems excellent, but it looks difficult for me to reproduce it for testing. Good luck with the rest of your work.
This pack is a patch add-on to RealRebelAI's node pack, so you must install his node pack first. This information is in the instructions.md file included with my pack.
You are also welcome to open discussions with more specific errors and requests at either my Github or HuggingFace repos:
jlucasmcrell/joyai-echo-multishot-patch: Multishot fixes + automation patch for the ComfyUI JoyAI-Echo GGUF node pack (LTX-2.3). Memory-bank trim fix, lip-sync fps fix, hires routing, AutoFinish masters.
joeygambino (Joey)
I am always happy to help, but I do need more information concerning what you're experiencing. Everyone's hardware and software versions are unique, so it's difficult to account for every minor difference.
@joeygambino Thanks for your reply.
However, I'm still getting the same errors whether I use gguf or safetensors with
I'm having trouble understanding the exceptions related to the model choice (should I choose gguf and rename it to safetensors?). Sorry... maybe I'm just too tired.
In any case, good luck with your project
@sdktertiaire2 You do not rename anything. I would have to see the specific errors you're receiving. It's all very much determined by your hardware limitations, ComfyUI version, etc.
If you can provide errors or screenshots in either of the repos mentioned by opening a discussion, I can be of much more help. There are already multiple discussions going on in those repos that may be of help as well.
@joeygambino ok i will try harder. I'll keep you posted. thanks
I was one of the people that really wanted this to work.Believe me. But unless i'm supposed to be doing something extravagant on top of a typical workflow, the facial consistency from the original picture for an image to video is not sustaining at all. The person's face morphs into the person fromYOUR videos and images. Not mine. I hope it gets better though!!
I'd honestly love to see how you accomplished that, as none of my images are embedded in the workflow. You can confirm that by doing a simple image search in the workflow folder. Feel free to open a discussion at HF or Github, I've talked several people through getting the workflow functioning properly.