
# songyu_ltx.23_mp4-rp8_rp5_v3

## 模型介绍

本模型依托魔搭社区（ModelScope）AIGC专区[模型训练](https://modelscope.cn/aigc/modelTraining)环境与算力完成训练。

* 模型类型：LoRA
* 基础模型：[Lightricks/LTX-2.3](https://modelscope.cn/models/Lightricks/LTX-2.3)
* 训练代码：[DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio)
* 训练数据量：8
* 总训练步数：2000
* 开源协议：Apache-2.0

## 推理代码

安装 [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio)：

```bash
pip install diffsynth
```

开始推理：

```python
import torch
from diffsynth.pipelines.ltx2_audio_video import LTX2AudioVideoPipeline, ModelConfig
from diffsynth.utils.data.media_io_ltx2 import write_video_audio_ltx2

vram_config = {
    "offload_dtype": torch.float8_e5m2,
    "offload_device": "cpu",
    "onload_dtype": torch.float8_e5m2,
    "onload_device": "cpu",
    "preparing_dtype": torch.float8_e5m2,
    "preparing_device": "cuda",
    "computation_dtype": torch.bfloat16,
    "computation_device": "cuda",
}
pipe = LTX2AudioVideoPipeline.from_pretrained(
    torch_dtype=torch.bfloat16,
    device="cuda",
    model_configs=[
        ModelConfig(model_id="google/gemma-3-12b-it-qat-q4_0-unquantized", origin_file_pattern="model-*.safetensors", **vram_config),
        ModelConfig(model_id="Lightricks/LTX-2.3", origin_file_pattern="ltx-2.3-22b-dev.safetensors", **vram_config),
        ModelConfig(model_id="Lightricks/LTX-2.3", origin_file_pattern="ltx-2.3-spatial-upscaler-x2-1.0.safetensors", **vram_config),
    ],
    tokenizer_config=ModelConfig(model_id="google/gemma-3-12b-it-qat-q4_0-unquantized"),
    stage2_lora_config=ModelConfig(model_id="Lightricks/LTX-2.3", origin_file_pattern="ltx-2.3-22b-distilled-lora-384.safetensors"),
    vram_limit=torch.cuda.mem_get_info("cuda")[1] / (1024 ** 3) - 0.5,
)
pipe.load_lora(pipe.dit, ModelConfig(model_id="wange1002/songyu_ltx.23_mp4-rp8_rp5_v3", origin_file_pattern="songyu_ltx.23_mp4-rp8_rp5_v3_c1-st2000.safetensors"))
prompt = "a cat"
video, audio = pipe(
    prompt=prompt,
    negative_prompt=pipe.default_negative_prompt["LTX-2.3"],
    height=1536, width=1024, num_frames=121,
    tiled=True, use_two_stage_pipeline=True,
)
write_video_audio_ltx2(video, audio, 'video.mp4', fps=24)
```


Files with SHA256: 81b1fa7c1bf5e829033e00fce5871d6d55af84685edfba281b2b9a6f0242bb76