Moss Yuni anime UWU base
This is a completely different model with reconfigured weight blocks. It handles details better and produces superior colors compared to the base 'moss' and 'SFT' models. However, it doesn't work very well with Turbo LoRAs. I have no desire to reconfigure it for Turbo, as I already have a Turbo version that works just fine.
but it also contains a dataset that was used to create the turbo version
INT8 will be with the assembled qkv (block atention) BF16 q+k+v (block atention)
STEPS
25-50 (optimal 25-30)
sampler
dpmpp 2m_SDE\ dpmpp SDE
scheduler
SGM uniform\simple
CFG 3-5
For those interested, here is the structure of MOSS SFT
MOSS checkpoint structure report
===========================================================
Input file: moss_image_sft_bf16.safetensors
Total keys: 379
Top key prefixes (first two components):
28 text_embedder.blocks.
8 layers.0.
8 layers.1.
8 layers.10.
8 layers.11.
8 layers.12.
8 layers.13.
8 layers.14.
8 layers.15.
8 layers.16.
8 layers.17.
8 layers.18.
8 layers.19.
8 layers.2.
8 layers.20.
8 layers.21.
8 layers.22.
8 layers.23.
8 layers.24.
8 layers.25.
8 layers.26.
8 layers.27.
8 layers.28.
8 layers.29.
8 layers.3.
8 layers.30.
8 layers.31.
8 layers.32.
8 layers.33.
8 layers.34.
8 layers.35.
8 layers.36.
8 layers.37.
8 layers.38.
8 layers.39.
8 layers.4.
8 layers.40.
8 layers.41.
8 layers.5.
8 layers.6.
MOSS linear prefix counts:
168 ''
Raw matched MOSS-like linear tensors:
total: 168
Raw part counts:
qkv : 42
q : 0
k : 0
v : 0
o : 42
ffn_up_gate : 42
ffn_down : 42
Detected blocks:
total blocks: 42
block numbers: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41]
Fusion estimate:
blocks with fused qkv key: 42
blocks with complete separate q/k/v: 0
blocks that can become qkv: 42
blocks with o: 42
blocks with ffn_up_gate: 42
blocks with ffn_down: 42
estimated linears after q/k/v fusion: 168
Missing expected parts by block:
missing qkv or complete q/k/v: []
missing o: []
missing ffn_up_gate: []
missing ffn_down: []
Blocks with incomplete separate q/k/v:
[]
Other / non-linear keys:
count: 211
Possible adapter-like keys:
none detected by simple markers
First 150 other keys:
final_linear_video.weight
final_norm_video.weight
layers.0.k_norm.weight
layers.0.norm1.weight
layers.0.norm2.weight
layers.0.q_norm.weight
layers.1.k_norm.weight
layers.1.norm1.weight
layers.1.norm2.weight
layers.1.q_norm.weight
layers.10.k_norm.weight
layers.10.norm1.weight
layers.10.norm2.weight
layers.10.q_norm.weight
layers.11.k_norm.weight
layers.11.norm1.weight
layers.11.norm2.weight
layers.11.q_norm.weight
layers.12.k_norm.weight
layers.12.norm1.weight
layers.12.norm2.weight
layers.12.q_norm.weight
layers.13.k_norm.weight
layers.13.norm1.weight
layers.13.norm2.weight
layers.13.q_norm.weight
layers.14.k_norm.weight
layers.14.norm1.weight
layers.14.norm2.weight
layers.14.q_norm.weight
layers.15.k_norm.weight
layers.15.norm1.weight
layers.15.norm2.weight
layers.15.q_norm.weight
layers.16.k_norm.weight
layers.16.norm1.weight
layers.16.norm2.weight
layers.16.q_norm.weight
layers.17.k_norm.weight
layers.17.norm1.weight
layers.17.norm2.weight
layers.17.q_norm.weight
layers.18.k_norm.weight
layers.18.norm1.weight
layers.18.norm2.weight
layers.18.q_norm.weight
layers.19.k_norm.weight
layers.19.norm1.weight
layers.19.norm2.weight
layers.19.q_norm.weight
layers.2.k_norm.weight
layers.2.norm1.weight
layers.2.norm2.weight
layers.2.q_norm.weight
layers.20.k_norm.weight
layers.20.norm1.weight
layers.20.norm2.weight
layers.20.q_norm.weight
layers.21.k_norm.weight
layers.21.norm1.weight
layers.21.norm2.weight
layers.21.q_norm.weight
layers.22.k_norm.weight
layers.22.norm1.weight
layers.22.norm2.weight
layers.22.q_norm.weight
layers.23.k_norm.weight
layers.23.norm1.weight
layers.23.norm2.weight
layers.23.q_norm.weight
layers.24.k_norm.weight
layers.24.norm1.weight
layers.24.norm2.weight
layers.24.q_norm.weight
layers.25.k_norm.weight
layers.25.norm1.weight
layers.25.norm2.weight
layers.25.q_norm.weight
layers.26.k_norm.weight
layers.26.norm1.weight
layers.26.norm2.weight
layers.26.q_norm.weight
layers.27.k_norm.weight
layers.27.norm1.weight
layers.27.norm2.weight
layers.27.q_norm.weight
layers.28.k_norm.weight
layers.28.norm1.weight
layers.28.norm2.weight
layers.28.q_norm.weight
layers.29.k_norm.weight
layers.29.norm1.weight
layers.29.norm2.weight
layers.29.q_norm.weight
layers.3.k_norm.weight
layers.3.norm1.weight
layers.3.norm2.weight
layers.3.q_norm.weight
layers.30.k_norm.weight
layers.30.norm1.weight
layers.30.norm2.weight
layers.30.q_norm.weight
layers.31.k_norm.weight
layers.31.norm1.weight
layers.31.norm2.weight
layers.31.q_norm.weight
layers.32.k_norm.weight
layers.32.norm1.weight
layers.32.norm2.weight
layers.32.q_norm.weight
layers.33.k_norm.weight
layers.33.norm1.weight
layers.33.norm2.weight
layers.33.q_norm.weight
layers.34.k_norm.weight
layers.34.norm1.weight
layers.34.norm2.weight
layers.34.q_norm.weight
layers.35.k_norm.weight
layers.35.norm1.weight
layers.35.norm2.weight
layers.35.q_norm.weight
layers.36.k_norm.weight
layers.36.norm1.weight
layers.36.norm2.weight
layers.36.q_norm.weight
layers.37.k_norm.weight
layers.37.norm1.weight
layers.37.norm2.weight
layers.37.q_norm.weight
layers.38.k_norm.weight
layers.38.norm1.weight
layers.38.norm2.weight
layers.38.q_norm.weight
layers.39.k_norm.weight
layers.39.norm1.weight
layers.39.norm2.weight
layers.39.q_norm.weight
layers.4.k_norm.weight
layers.4.norm1.weight
layers.4.norm2.weight
layers.4.q_norm.weight
layers.40.k_norm.weight
layers.40.norm1.weight
layers.40.norm2.weight
layers.40.q_norm.weight
layers.41.k_norm.weight
layers.41.norm1.weight
layers.41.norm2.weight
layers.41.q_norm.weight
... 61 more
structure of MOSS
Input file: moss_image_base_bf16.safetensors
Total keys: 379
Top key prefixes (first two components):
28 text_embedder.blocks.
8 layers.0.
8 layers.1.
8 layers.10.
8 layers.11.
8 layers.12.
8 layers.13.
8 layers.14.
8 layers.15.
8 layers.16.
8 layers.17.
8 layers.18.
8 layers.19.
8 layers.2.
8 layers.20.
8 layers.21.
8 layers.22.
8 layers.23.
8 layers.24.
8 layers.25.
8 layers.26.
8 layers.27.
8 layers.28.
8 layers.29.
8 layers.3.
8 layers.30.
8 layers.31.
8 layers.32.
8 layers.33.
8 layers.34.
8 layers.35.
8 layers.36.
8 layers.37.
8 layers.38.
8 layers.39.
8 layers.4.
8 layers.40.
8 layers.41.
8 layers.5.
8 layers.6.
MOSS linear prefix counts:
168 ''
Raw matched MOSS-like linear tensors:
total: 168
Raw part counts:
qkv : 42
q : 0
k : 0
v : 0
o : 42
ffn_up_gate : 42
ffn_down : 42
Detected blocks:
total blocks: 42
block numbers: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41]
Fusion estimate:
blocks with fused qkv key: 42
blocks with complete separate q/k/v: 0
blocks that can become qkv: 42
blocks with o: 42
blocks with ffn_up_gate: 42
blocks with ffn_down: 42
estimated linears after q/k/v fusion: 168
Missing expected parts by block:
missing qkv or complete q/k/v: []
missing o: []
missing ffn_up_gate: []
missing ffn_down: []
Blocks with incomplete separate q/k/v:
[]
Other / non-linear keys:
count: 211
Possible adapter-like keys:
none detected by simple markers
First 150 other keys:
final_linear_video.weight
final_norm_video.weight
layers.0.k_norm.weight
layers.0.norm1.weight
layers.0.norm2.weight
layers.0.q_norm.weight
layers.1.k_norm.weight
layers.1.norm1.weight
layers.1.norm2.weight
layers.1.q_norm.weight
layers.10.k_norm.weight
layers.10.norm1.weight
layers.10.norm2.weight
layers.10.q_norm.weight
layers.11.k_norm.weight
layers.11.norm1.weight
layers.11.norm2.weight
layers.11.q_norm.weight
layers.12.k_norm.weight
layers.12.norm1.weight
layers.12.norm2.weight
layers.12.q_norm.weight
layers.13.k_norm.weight
layers.13.norm1.weight
layers.13.norm2.weight
layers.13.q_norm.weight
layers.14.k_norm.weight
layers.14.norm1.weight
layers.14.norm2.weight
layers.14.q_norm.weight
layers.15.k_norm.weight
layers.15.norm1.weight
layers.15.norm2.weight
layers.15.q_norm.weight
layers.16.k_norm.weight
layers.16.norm1.weight
layers.16.norm2.weight
layers.16.q_norm.weight
layers.17.k_norm.weight
layers.17.norm1.weight
layers.17.norm2.weight
layers.17.q_norm.weight
layers.18.k_norm.weight
layers.18.norm1.weight
layers.18.norm2.weight
layers.18.q_norm.weight
layers.19.k_norm.weight
layers.19.norm1.weight
layers.19.norm2.weight
layers.19.q_norm.weight
layers.2.k_norm.weight
layers.2.norm1.weight
layers.2.norm2.weight
layers.2.q_norm.weight
layers.20.k_norm.weight
layers.20.norm1.weight
layers.20.norm2.weight
layers.20.q_norm.weight
layers.21.k_norm.weight
layers.21.norm1.weight
layers.21.norm2.weight
layers.21.q_norm.weight
layers.22.k_norm.weight
layers.22.norm1.weight
layers.22.norm2.weight
layers.22.q_norm.weight
layers.23.k_norm.weight
layers.23.norm1.weight
layers.23.norm2.weight
layers.23.q_norm.weight
layers.24.k_norm.weight
layers.24.norm1.weight
layers.24.norm2.weight
layers.24.q_norm.weight
layers.25.k_norm.weight
layers.25.norm1.weight
layers.25.norm2.weight
layers.25.q_norm.weight
layers.26.k_norm.weight
layers.26.norm1.weight
layers.26.norm2.weight
layers.26.q_norm.weight
layers.27.k_norm.weight
layers.27.norm1.weight
layers.27.norm2.weight
layers.27.q_norm.weight
layers.28.k_norm.weight
layers.28.norm1.weight
layers.28.norm2.weight
layers.28.q_norm.weight
layers.29.k_norm.weight
layers.29.norm1.weight
layers.29.norm2.weight
layers.29.q_norm.weight
layers.3.k_norm.weight
layers.3.norm1.weight
layers.3.norm2.weight
layers.3.q_norm.weight
layers.30.k_norm.weight
layers.30.norm1.weight
layers.30.norm2.weight
layers.30.q_norm.weight
layers.31.k_norm.weight
layers.31.norm1.weight
layers.31.norm2.weight
layers.31.q_norm.weight
layers.32.k_norm.weight
layers.32.norm1.weight
layers.32.norm2.weight
layers.32.q_norm.weight
layers.33.k_norm.weight
layers.33.norm1.weight
layers.33.norm2.weight
layers.33.q_norm.weight
layers.34.k_norm.weight
layers.34.norm1.weight
layers.34.norm2.weight
layers.34.q_norm.weight
layers.35.k_norm.weight
layers.35.norm1.weight
layers.35.norm2.weight
layers.35.q_norm.weight
layers.36.k_norm.weight
layers.36.norm1.weight
layers.36.norm2.weight
layers.36.q_norm.weight
layers.37.k_norm.weight
layers.37.norm1.weight
layers.37.norm2.weight
layers.37.q_norm.weight
layers.38.k_norm.weight
layers.38.norm1.weight
layers.38.norm2.weight
layers.38.q_norm.weight
layers.39.k_norm.weight
layers.39.norm1.weight
layers.39.norm2.weight
layers.39.q_norm.weight
layers.4.k_norm.weight
layers.4.norm1.weight
layers.4.norm2.weight
layers.4.q_norm.weight
layers.40.k_norm.weight
layers.40.norm1.weight
layers.40.norm2.weight
layers.40.q_norm.weight
layers.41.k_norm.weight
layers.41.norm1.weight
layers.41.norm2.weight
layers.41.q_norm.weight
... 61 more
As for me, it’s the following:
MOSS checkpoint structure report
===========================================================
Input file: moss anime UwU base .safetensors
Total keys: 462
Top key prefixes (first two components):
462 model.diffusion_model.
MOSS linear prefix counts:
252 'model.diffusion_model.'
Raw matched MOSS-like linear tensors:
total: 252
Raw part counts:
qkv : 0
q : 42
k : 42
v : 42
o : 42
ffn_up_gate : 42
ffn_down : 42
Detected blocks:
total blocks: 42
block numbers: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41]
Fusion estimate:
blocks with fused qkv key: 0
blocks with complete separate q/k/v: 42
blocks that can become qkv: 42
blocks with o: 42
blocks with ffn_up_gate: 42
blocks with ffn_down: 42
estimated linears after q/k/v fusion: 168
Missing expected parts by block:
missing qkv or complete q/k/v: []
missing o: []
missing ffn_up_gate: []
missing ffn_down: []
Blocks with incomplete separate q/k/v:
[]
Other / non-linear keys:
count: 210
Possible adapter-like keys:
none detected by simple markers
First 150 other keys:
model.diffusion_model.final_linear_video.weight
model.diffusion_model.final_norm_video.weight
model.diffusion_model.layers.0.k_norm.weight
model.diffusion_model.layers.0.norm1.weight
model.diffusion_model.layers.0.norm2.weight
model.diffusion_model.layers.0.q_norm.weight
model.diffusion_model.layers.1.k_norm.weight
model.diffusion_model.layers.1.norm1.weight
model.diffusion_model.layers.1.norm2.weight
model.diffusion_model.layers.1.q_norm.weight
model.diffusion_model.layers.10.k_norm.weight
model.diffusion_model.layers.10.norm1.weight
model.diffusion_model.layers.10.norm2.weight
model.diffusion_model.layers.10.q_norm.weight
model.diffusion_model.layers.11.k_norm.weight
model.diffusion_model.layers.11.norm1.weight
model.diffusion_model.layers.11.norm2.weight
model.diffusion_model.layers.11.q_norm.weight
model.diffusion_model.layers.12.k_norm.weight
model.diffusion_model.layers.12.norm1.weight
model.diffusion_model.layers.12.norm2.weight
model.diffusion_model.layers.12.q_norm.weight
model.diffusion_model.layers.13.k_norm.weight
model.diffusion_model.layers.13.norm1.weight
model.diffusion_model.layers.13.norm2.weight
model.diffusion_model.layers.13.q_norm.weight
model.diffusion_model.layers.14.k_norm.weight
model.diffusion_model.layers.14.norm1.weight
model.diffusion_model.layers.14.norm2.weight
model.diffusion_model.layers.14.q_norm.weight
model.diffusion_model.layers.15.k_norm.weight
model.diffusion_model.layers.15.norm1.weight
model.diffusion_model.layers.15.norm2.weight
model.diffusion_model.layers.15.q_norm.weight
model.diffusion_model.layers.16.k_norm.weight
model.diffusion_model.layers.16.norm1.weight
model.diffusion_model.layers.16.norm2.weight
model.diffusion_model.layers.16.q_norm.weight
model.diffusion_model.layers.17.k_norm.weight
model.diffusion_model.layers.17.norm1.weight
model.diffusion_model.layers.17.norm2.weight
model.diffusion_model.layers.17.q_norm.weight
model.diffusion_model.layers.18.k_norm.weight
model.diffusion_model.layers.18.norm1.weight
model.diffusion_model.layers.18.norm2.weight
model.diffusion_model.layers.18.q_norm.weight
model.diffusion_model.layers.19.k_norm.weight
model.diffusion_model.layers.19.norm1.weight
model.diffusion_model.layers.19.norm2.weight
model.diffusion_model.layers.19.q_norm.weight
model.diffusion_model.layers.2.k_norm.weight
model.diffusion_model.layers.2.norm1.weight
model.diffusion_model.layers.2.norm2.weight
model.diffusion_model.layers.2.q_norm.weight
model.diffusion_model.layers.20.k_norm.weight
model.diffusion_model.layers.20.norm1.weight
model.diffusion_model.layers.20.norm2.weight
model.diffusion_model.layers.20.q_norm.weight
model.diffusion_model.layers.21.k_norm.weight
model.diffusion_model.layers.21.norm1.weight
model.diffusion_model.layers.21.norm2.weight
model.diffusion_model.layers.21.q_norm.weight
model.diffusion_model.layers.22.k_norm.weight
model.diffusion_model.layers.22.norm1.weight
model.diffusion_model.layers.22.norm2.weight
model.diffusion_model.layers.22.q_norm.weight
model.diffusion_model.layers.23.k_norm.weight
model.diffusion_model.layers.23.norm1.weight
model.diffusion_model.layers.23.norm2.weight
model.diffusion_model.layers.23.q_norm.weight
model.diffusion_model.layers.24.k_norm.weight
model.diffusion_model.layers.24.norm1.weight
model.diffusion_model.layers.24.norm2.weight
model.diffusion_model.layers.24.q_norm.weight
model.diffusion_model.layers.25.k_norm.weight
model.diffusion_model.layers.25.norm1.weight
model.diffusion_model.layers.25.norm2.weight
model.diffusion_model.layers.25.q_norm.weight
model.diffusion_model.layers.26.k_norm.weight
model.diffusion_model.layers.26.norm1.weight
model.diffusion_model.layers.26.norm2.weight
model.diffusion_model.layers.26.q_norm.weight
model.diffusion_model.layers.27.k_norm.weight
model.diffusion_model.layers.27.norm1.weight
model.diffusion_model.layers.27.norm2.weight
model.diffusion_model.layers.27.q_norm.weight
model.diffusion_model.layers.28.k_norm.weight
model.diffusion_model.layers.28.norm1.weight
model.diffusion_model.layers.28.norm2.weight
model.diffusion_model.layers.28.q_norm.weight
model.diffusion_model.layers.29.k_norm.weight
model.diffusion_model.layers.29.norm1.weight
model.diffusion_model.layers.29.norm2.weight
model.diffusion_model.layers.29.q_norm.weight
model.diffusion_model.layers.3.k_norm.weight
model.diffusion_model.layers.3.norm1.weight
model.diffusion_model.layers.3.norm2.weight
model.diffusion_model.layers.3.q_norm.weight
model.diffusion_model.layers.30.k_norm.weight
model.diffusion_model.layers.30.norm1.weight
model.diffusion_model.layers.30.norm2.weight
model.diffusion_model.layers.30.q_norm.weight
model.diffusion_model.layers.31.k_norm.weight
model.diffusion_model.layers.31.norm1.weight
model.diffusion_model.layers.31.norm2.weight
model.diffusion_model.layers.31.q_norm.weight
model.diffusion_model.layers.32.k_norm.weight
model.diffusion_model.layers.32.norm1.weight
model.diffusion_model.layers.32.norm2.weight
model.diffusion_model.layers.32.q_norm.weight
model.diffusion_model.layers.33.k_norm.weight
model.diffusion_model.layers.33.norm1.weight
model.diffusion_model.layers.33.norm2.weight
model.diffusion_model.layers.33.q_norm.weight
model.diffusion_model.layers.34.k_norm.weight
model.diffusion_model.layers.34.norm1.weight
model.diffusion_model.layers.34.norm2.weight
model.diffusion_model.layers.34.q_norm.weight
model.diffusion_model.layers.35.k_norm.weight
model.diffusion_model.layers.35.norm1.weight
model.diffusion_model.layers.35.norm2.weight
model.diffusion_model.layers.35.q_norm.weight
model.diffusion_model.layers.36.k_norm.weight
model.diffusion_model.layers.36.norm1.weight
model.diffusion_model.layers.36.norm2.weight
model.diffusion_model.layers.36.q_norm.weight
model.diffusion_model.layers.37.k_norm.weight
model.diffusion_model.layers.37.norm1.weight
model.diffusion_model.layers.37.norm2.weight
model.diffusion_model.layers.37.q_norm.weight
model.diffusion_model.layers.38.k_norm.weight
model.diffusion_model.layers.38.norm1.weight
model.diffusion_model.layers.38.norm2.weight
model.diffusion_model.layers.38.q_norm.weight
model.diffusion_model.layers.39.k_norm.weight
model.diffusion_model.layers.39.norm1.weight
model.diffusion_model.layers.39.norm2.weight
model.diffusion_model.layers.39.q_norm.weight
model.diffusion_model.layers.4.k_norm.weight
model.diffusion_model.layers.4.norm1.weight
model.diffusion_model.layers.4.norm2.weight
model.diffusion_model.layers.4.q_norm.weight
model.diffusion_model.layers.40.k_norm.weight
model.diffusion_model.layers.40.norm1.weight
model.diffusion_model.layers.40.norm2.weight
model.diffusion_model.layers.40.q_norm.weight
model.diffusion_model.layers.41.k_norm.weight
model.diffusion_model.layers.41.norm1.weight
model.diffusion_model.layers.41.norm2.weight
model.diffusion_model.layers.41.q_norm.weight
... 60 more
Moss Yuni anime UWU 8 step
sampler
dpmpp 2m_SDE\ er_sde
scheduler
SGM uniform\bong_tangent\beta 57
CFG 1-1.1
DEMO comparison
MOSS SFT + 8-step lora..............................................................................MOSS Yuni Anime UwU turbo
