[Massed Compute / RunPod / Any Cloud GPU Linux VM] How to Install SageAttention for ComfyUI

Since it was a pain in the ass for me, I thought I'd share how I installed Sage. This guide outlines how to install SageAttention for ComfyUI on a cloud GPU Linux VM, such as Massed Compute or RunPod. It’s tailored for Ubuntu-based systems with NVIDIA GPUs (e.g., L40, compute capability 8.9) and addresses common compiler issues encountered during installation.

This guide assumes you already have ComfyUI Installed and are running Linux or Linux via VM.

[Tested on: - Ubuntu 22.04 - NVIDIA L40 GPU (48GB vRAM) - CUDA 12.8 - Python 3.10 ]

Prerequisites

• Check System Basics:

lsb_release -a # Confirm Ubuntu version (e.g., 22.04) 
nvidia-smi # Verify GPU and CUDA version (e.g., 12.8) 
python3 --version # Ensure Python 3.10 or higher

• Install CUDA Toolkit:

  • Download from NVIDIA CUDA Downloads

  • Example for Ubuntu 22.04, CUDA 12.8:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin 

sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub 

sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /" 

sudo apt-get update sudo apt-get install -y cuda

• Add To PATH

echo 'export PATH=/usr/local/cuda-12.8/bin:$PATH' >> ~/.bashrc 
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.8/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc 
source ~/.bashrc 

Install SageAttention

SageAttention requires compilation with CUDA and a compatible C++ compiler. The default `gcc` on some VMs (e.g., `gcc-12`) may cause issues, so we force it to use `gcc-11` / `g++-11`.

Step 1: Install Build Tools

bash sudo apt-get update sudo apt-get install -y build-essential g++-11 gcc-11 

Step 2: Verify Compiler

g++-11 --version # Should show 11.4.0 or similar 
find /usr/lib -name cc1plus # Should find /usr/lib/gcc/x86_64-linux-gnu/11/cc1plus

Step 3: Set Compiler Environment

bash export CC=/usr/bin/gcc-11 export CXX=/usr/bin/g++-11

Step 4: Clone and Install SageAttention

cd ~/apps # Or your preferred directory 
git clone https://github.com/thu-ml/SageAttention.git 
cd SageAttention rm -rf build dist *.egg-info # Clean any prior failed builds 
pip3 install numpy # Fix potential NumPy dependency 
pip3 install -e . # Install in editable mode

Step 5: Verify Installation

python3 -c "import sageattention; print('SageAttention installed successfully')"

Integrate with ComfyUI

1. **Start ComfyUI:**: passing the `--use-sage-attention` argument

```bash cd ~/ComfyUI source venv/bin/activate # If using a virtual env python3 main.py --listen --use-sage-attention ```

2. **SCheck Usage:**: SageAttention should auto-optimize attention layers. Look for performance gains (e.g., 2.1-3.1x speedup per the repo) or add it explicitly to workflows (see [SageAttention README](https://github.com/thu-ml/SageAttention#how-to-use)).

Troubleshooting

- Compiler Error: (cannot execute `cc1plus`):

• Ensure `g++-11` is installed and `CC`/`CXX` are set.

• Alternative: Edit `SageAttention/setup.py`, add `extra_compile_args={'nvcc': ['-ccbin', '/usr/bin/g++-11']}` to `CUDAExtension`.

- CUDA Mismatch Warning: Ignore if minor (e.g., PyTorch 12.4 vs CUDA 12.8); reinstall PyTorch with matching CUDA if needed.

- VM Support:: Contact your provider (e.g., Massed Compute) if compiler paths seem off. # Notes - Tested on Massed Compute with an L40 GPU, but should work on RunPod or similar cloud VMs with NVIDIA GPUs.

- Adjust CUDA version (e.g., 12.1 instead of 12.8) in PyTorch install if compatibility issues arise.

<Edit History: 3/27/2025>