Hunyuan... How do I prepare video datasets for this?

My nearly prepared for release captioning software, is a software already fully capable of handling this sort of thing; but I'd like to streamline this process before release.

It can slice, downscale, upscale, interpolate frames, superimpose frame differences, and many more details.

I can generate latents if I am pointed at some hunyuan based softwares that use latents so I can get the latent parameters and so on. Latents are much smaller and are bucketed.

I can set up a video bucketing system, a video interpolation system, a blur differentiation system between similar frames, frame flip, and more.

I can set it up to multi-caption video stills using my nsfw T5 small, that thing can caption hundreds of things a second and it only takes about 3 gigs vram... it's really dumb though, and only has a 200 token window.

I can have it so it snips whole videos into second segments, or leave them whole and scale, or whatever.

So, I thought about doing it all right; kind of just leaving it open ended and letting people do what they want, but that wouldn't be very ease of access. The software is already technical enough without a GUI, so I really need to work on streamlining certain elements.

Hunyuan training styles being a useful one.

If anyone can give some solid advice on this sort of thing, I can get started on the video slicer, downscaler, and so on. I know most people won't be able to train big videos, and I know it takes a long time for me to even generate big videos on a 4090 so I assume the majority of trainings will be between 128x128 and 512x512 just for the sake of hardware limitations on average.

I'll likely do a 768x768 train but that's because I'm a nut, and nobody will want it.

Advice, guides, tips, help, anything. Give me the info, anything to make this faster, and the hunyuan scripts will drop first.

If that happens, you better believe the professional-grade hunyuan lora surge will happen. I'll just latch an accelerate trainer straight into those scripts with choice params that I update regularly after testing.
I'm interested, and there's not much tech for it yet. Latching my pure high-end caption software straight to this sort of thing is just good timing.

I'm pretty careful with exception catching, but there's some that will need a bit more attention; like pytorch running out of vram for example. Can't dodge the hardware not being good enough. Can definitely make good exception checks though.