What do I do when an LLM image generator malfunctions?

I make a LoRA of it.

Yes, the is a SDXL model, not a Z Image Base model. It's trained on Z Image Base generations.