<h2 id="these-are-not-int8-convrot.">These are not INT8 ConvRot.</h2><p>These models are INT8 Tensorwise. if you are on <span style="color:rgb(64, 192, 87)"><strong>AMD ROCm</strong></span>, these models will pack a real punch if you take the time to custom build ComfyUI with <a rel="ugc" href="https://github.com/ROCm/flash-attention">ROCm\FlashAttention</a> + AMD Triton / AITER... But they are expected to pack <span style="color:rgb(64, 192, 87)"><strong>a performance boost on Nvidia</strong></span> cards too.</p><h3 id="minimum-start-up-parameters-for-rocmflashattention-+-amd-triton:"><br />Minimum start up parameters for ROCm\FlashAttention + AMD Triton:</h3><p>IMPORTANT: Do not install flashattention offered by pip get official <a rel="ugc" href="https://github.com/ROCm/flash-attention">ROCm\flashattention from github.</a></p><pre><code>export FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE

python main.py \
  --use-flash-attention \
  --disable-xformers </code></pre><p><br />More details and benchmarks coming soon.</p>

Files with SHA256: d9edf40649b0c00a0d623d78c67e87ab7eceac7bd3dd1995b97f64f493665fc9