这个链接下包含 STEPFUN-AI 发布的 ‌ACEStep‌ &amp; Step1XEdit5.11 updated ACESTEP Audio2Audio workflows<h1 id="acestep-music-generation-2c7dnm3ge">ACEStep‌ Music Generation</h1><img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/89884520-6747-4472-9ef5-41edf87683be/width=525/89884520-6747-4472-9ef5-41edf87683be.jpeg" />We introduce ACE-Step, a novel open-source foundation model for music generation that overcomes key limitations of existing approaches and achieves state-of-the-art performance through a holistic architectural design. Current methods face inherent trade-offs between generation speed, musical coherence, and controllability. For instance, LLM-based models (e.g., Yue, SongGen) excel at lyric alignment but suffer from slow inference and structural artifacts. Diffusion models (e.g., DiffRhythm), on the other hand, enable faster synthesis but often lack long-range structural coherence.<a target="_blank" rel="ugc" href="https://github.com/ace-step/ACE-Step">‌ACEStep‌</a> 是一款由 ACE Studio 与 StepFun 联合开发的音乐生成模型，被誉为“音乐界的Stable Diffusion”。该模型以其惊人的生成速度和多样化功能引发行业热议，能够在短短20秒内生成一首长达4分钟的完整音乐作品，效率比主流模型快15倍以上‌ACE-Step bridges this gap by integrating diffusion-based generation with Sana’s Deep Compression AutoEncoder (DCAE) and a lightweight linear transformer. It further leverages MERT and m-hubert to align semantic representations (REPA) during training, enabling rapid convergence. As a result, our model synthesizes up to 4 minutes of music in just 20 seconds on an A100 GPU—15× faster than LLM-based baselines—while achieving superior musical coherence and lyric alignment across melody, harmony, and rhythm metrics. Moreover, ACE-Step preserves fine-grained acoustic details, enabling advanced control mechanisms such as voice cloning, lyric editing, remixing, and track generation (e.g., lyric2vocal, singing2accompaniment).<a target="_blank" rel="ugc" href="https://private-user-images.githubusercontent.com/30956809/441739923-0a13d90a-9086-47ee-abab-976bad20fa7c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDY4NjA4MzQsIm5iZiI6MTc0Njg2MDUzNCwicGF0aCI6Ii8zMDk1NjgwOS80NDE3Mzk5MjMtMGExM2Q5MGEtOTA4Ni00N2VlLWFiYWItOTc2YmFkMjBmYTdjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA1MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNTEwVDA3MDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTE0ZDg4ZGIyYzVmMzg1MTc3ZTAzNmE1Y2EwNjMyZGY3MDBlMWY2MDRkMGVlMmQ5ZGMwMDUzZjExM2E2ZDgyMmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.APSO1yodIjmcoLizCyIsvogv9cOQvokgD0prFJ4Dkis"><img src="https://private-user-images.githubusercontent.com/30956809/441739923-0a13d90a-9086-47ee-abab-976bad20fa7c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDY4NjA4MzQsIm5iZiI6MTc0Njg2MDUzNCwicGF0aCI6Ii8zMDk1NjgwOS80NDE3Mzk5MjMtMGExM2Q5MGEtOTA4Ni00N2VlLWFiYWItOTc2YmFkMjBmYTdjLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA1MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwNTEwVDA3MDIxNFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTE0ZDg4ZGIyYzVmMzg1MTc3ZTAzNmE1Y2EwNjMyZGY3MDBlMWY2MDRkMGVlMmQ5ZGMwMDUzZjExM2E2ZDgyMmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.APSO1yodIjmcoLizCyIsvogv9cOQvokgD0prFJ4Dkis" alt="图片" /></a>技术特点和应用场景ACEStep 基于 DiT（Diffusion Transformer）架构，采用轻量化设计，支持在消费级GPU上运行，显著降低了硬件门槛。该模型支持19种语言，包括英语、中文、日语、西班牙语等，为全球音乐创作者提供了广泛的适用性‌ACEStep的核心功能包括：‌歌词驱动创作‌：输入歌词，模型自动生成旋律并演唱完整歌曲。‌风格化编曲‌：根据指定的风格标签（如说唱、电子乐、流行等），自动生成伴奏和配器。‌精准修改‌：支持对歌曲某段歌词进行修改，而不影响原有旋律的连贯性。‌多样化生成‌：能够生成带风格的说唱、电子乐、人声或复杂配器，满足不同音乐场景需求‌<h3 id="baseline-quality-g8yj91kdk">🎯 Baseline Quality</h3>🌈 Diverse Styles &amp; Genres<ul><li>🎸 Supports all mainstream music styles with various description formats including short tags, descriptive text, or use-case scenarios</li><li>🎷 Capable of generating music across different genres with appropriate instrumentation and style</li></ul>🌍 Multiple Languages<ul><li>🗣️ Supports 19 languages with top 10 well-performing languages including:<ul><li>🇺🇸 English, 🇨🇳 Chinese, 🇷🇺 Russian, 🇪🇸 Spanish, 🇯🇵 Japanese, 🇩🇪 German, 🇫🇷 French, 🇵🇹 Portuguese, 🇮🇹 Italian, 🇰🇷 Korean</li></ul></li><li>⚠️ Due to data imbalance, less common languages may underperform</li></ul>🎻 Instrumental Styles<ul><li>🎹 Supports various instrumental music generation across different genres and styles</li><li>🎺 Capable of producing realistic instrumental tracks with appropriate timbre and expression for each instrument</li><li>🎼 Can generate complex arrangements with multiple instruments while maintaining musical coherence</li></ul>🎤 Vocal Techniques<ul><li>🎙️ Capable of rendering various vocal styles and techniques with good quality</li><li>🗣️ Supports different vocal expressions including various singing techniques and styles</li></ul>Rather than building yet another end-to-end text-to-music pipeline, our vision is to establish a foundation model for music AI: a fast, general-purpose, efficient yet flexible architecture that makes it easy to train sub-tasks on top of it. This paves the way for developing powerful tools that seamlessly integrate into the creative workflows of music artists, producers, and content creators. In short, we aim to build the Stable Diffusion moment for music.最新动态和未来发展前景ACEStep 通过GitHub（<a target="_blank" rel="ugc" href="https://github.com/stepfun-ai/Step-Audio">stepfun-ai/Step-Audio</a>）向开发者开放，允许社区对其进行微调以适配多样化的音乐任务。<h2 id="license-and-disclaimer-4tcy1zx9e">📜 License &amp; Disclaimer</h2>This project is licensed under <a target="_blank" rel="ugc" href="https://github.com/ace-step/ACE-Step/blob/main/LICENSE">Apache License 2.0</a>ACE-Step enables original music generation across diverse genres, with applications in creative production, education, and entertainment. While designed to support positive and artistic use cases, we acknowledge potential risks such as unintentional copyright infringement due to stylistic similarity, inappropriate blending of cultural elements, and misuse for generating harmful content. To ensure responsible use, we encourage users to verify the originality of generated works, clearly disclose AI involvement, and obtain appropriate permissions when adapting protected styles or materials. By using ACE-Step, you agree to uphold these principles and respect artistic integrity, cultural diversity, and legal compliance. The authors are not responsible for any misuse of the model, including but not limited to copyright violations, cultural insensitivity, or the generation of harmful content.🔔 Important Notice The only official website for the ACE-Step project is our GitHub Pages site. We do not operate any other websites. 🚫 Fake domains include but are not limited to: ac**<a target="_blank" rel="ugc" href="http://p.com">p.com</a>, a**<a target="_blank" rel="ugc" href="http://p.org">p.org</a>, a***<a target="_blank" rel="ugc" href="http://c.org">c.org</a> ⚠️ Please be cautious. Do not visit, trust, or make payments on any of those sites.<h2 id="acknowledgements-8yqrg6izz">🙏 Acknowledgements</h2>This project is co-led by ACE Studio and StepFun.<h2 id="citation-ugxws9u3d">📖 Citation</h2>If you find this project useful for your research, please consider citing:<pre><code>@misc{gong2025acestep,
	title={ACE-Step: A Step Towards Music Generation Foundation Model},
	author={Junmin Gong, Wenxiao Zhao, Sen Wang, Shengyuan Xu, Jing Guo}, 
	howpublished={\url{https://github.com/ace-step/ACE-Step}},
	year={2025},
	note={GitHub repository}
}</code></pre><hr /><h1 id="step1xedit-image-editing-model-kricchwaf">Step1XEdit image editing model</h1><a target="_blank" rel="ugc" href="https://github.com/stepfun-ai/Step-Audio/blob/main/assets/logo.png"><img src="https://github.com/stepfun-ai/Step-Audio/raw/main/assets/logo.png" /></a>RED版节点包及示例工作流，支持 Step1XEdit，自定义注意力机制，16G 显存占用。RED nodes package and sample workflow, supports Step1XEdit, custom attention mechanism, and occupies 16GB of video memory.此自定义节点将 <a target="_blank" rel="ugc" href="https://github.com/stepfun-ai/Step1X-Edit">Step1X-Edit</a> 图像编辑模型集成到 <a target="_blank" rel="ugc" href="https://github.com/comfyanonymous/ComfyUI">ComfyUI</a> 中。Step1X-Edit 是一个先进的图像编辑模型，它接收参考图像和用户的编辑指令，生成新的图像。RED nodes for Step1XEdit, custom attention mechanism with 16GB of video memory usage. This custom node integrates the [Step1X-Edit] image editing model into [ComfyUI]. Step1X-Edit is a state-of-the-art image editing model that processes a reference image and user's editing instruction to generate a new image.Step1X-Edit 基于FLUX 模型规格，但是替换了文本编码器为Qwen2.5-VL 7B模型权重为重新训练，与BSL的 FLUX.1 授权协议没有继承关系 License:apache-2.0<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/1c35249a-3333-4941-be23-4607faf2b900/width=525/1c35249a-3333-4941-be23-4607faf2b900.jpeg" /><h3 id="-24y7c05oo">主要特性</h3><ul><li>支持多种注意力实现方式(Flash Attention 2, PyTorch SDPA, Vanilla)</li><li>灵活配置以适应不同硬件能力</li><li>优化性能和兼容性</li></ul>功能特点<ul><li>支持 FP8 推理</li><li>支持自定义注意力实现(Flash/PyTorch(SDPA)/Vanilla)</li><li>无需折腾安装Flash Attention2(如有需要可<a target="_blank" rel="ugc" href="https://github.com/kingbri1/flash-attention/releases">自行安装</a>)</li></ul>代码仓库<a target="_blank" rel="ugc" href="https://github.com/QijiTec/ComfyUI-RED-Step1X-Edit">https://github.com/QijiTec/ComfyUI-RED-Step1X-Edit</a>forked from <a target="_blank" rel="ugc" href="https://github.com/raykindle/ComfyUI_Step1X-Edit">raykindle/ComfyUI_Step1X-Edit</a>🔥 原作者新加入了Teacache加速器支持（需要Flash-attn2）模型仓库（CivitAI 页面可以直接下载）<ul><li>Step1X-Edit 扩散模型：从 <a target="_blank" rel="ugc" href="https://huggingface.co/meimeilook/Step1X-Edit-FP8/tree/main">HuggingFace</a> 下载 <code>step1x-edit-i1258-FP8.safetensors</code> 并放置在 ComfyUI 的 <code>models/diffusion_models</code> 目录中</li><li>Step1X-Edit VAE：从 <a target="_blank" rel="ugc" href="https://huggingface.co/meimeilook/Step1X-Edit-FP8/tree/main">HuggingFace</a> 下载 <code>vae.safetensors</code> 并放置在 ComfyUI 的 <code>models/vae</code> 目录中 （等同于FLUX diffusers 版VAE）</li><li>Qwen2.5-VL 模型：下载 <a target="_blank" rel="ugc" href="https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/tree/main">Qwen2.5-VL-7B-Instruct</a> 并放置在 ComfyUI 的 <code>models/text_encoders/Qwen2.5-VL-7B-Instruct</code> 目录中</li></ul>Download Step1X-Edit-FP8 model<ul><li>Step1X-Edit diffusion model: Download <code>step1x-edit-i1258-FP8.safetensors</code> from <a target="_blank" rel="ugc" href="https://huggingface.co/meimeilook/Step1X-Edit-FP8/tree/main">HuggingFace</a> and place it in ComfyUI's <code>models/diffusion_models</code> directory</li><li>Step1X-Edit VAE: Download <code>vae.safetensors</code> from <a target="_blank" rel="ugc" href="https://huggingface.co/meimeilook/Step1X-Edit-FP8/tree/main">HuggingFace</a> and place it in ComfyUI's <code>models/vae</code> directory</li><li>Qwen2.5-VL model: Download <a target="_blank" rel="ugc" href="https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/tree/main">Qwen2.5-VL-7B-Instruct</a> and place it in ComfyUI's <code>models/text_encoders/Qwen2.5-VL-7B-Instruct</code> directory</li></ul>Configure the model parameters<ul><li>Select <code>step1x-edit-i1258-FP8.safetensors</code> as the diffusion model</li><li>Select <code>vae.safetensors</code> as the VAE</li><li>Set <code>Qwen2.5-VL-7B-Instruct</code> as the text encoder</li><li>Set additional parameters (<code>dtype</code>, <code>quantized</code>, <code>offload</code>) as needed</li></ul><h2 id="stepfun-aistep1x-edit-yue4gqazb">🔥🔥🔥🔥🔥🔥 stepfun-ai/Step1X-Edit🔥🔥🔥🔥🔥🔥</h2><img src="https://huggingface.co/stepfun-ai/Step1X-Edit/resolve/main/assets/image_edit_demo.gif" alt="demo" />Step1X-Edit: a unified image editing model performs impressively on various genuine user instructions.<h2 id="model-introduction-atslppux0">Model introduction</h2><img src="https://huggingface.co/stepfun-ai/Step1X-Edit/resolve/main/assets/arch.png" alt="demo" />Framework of Step1X-Edit. Step1X-Edit leverages the image understanding capabilities of MLLMs to parse editing instructions and generate editing tokens, which are then decoded into images using a DiT-based network.More details please refer to our <a target="_blank" rel="ugc" href="https://arxiv.org/abs/2504.17761">technical report</a>.<h2 id="benchmark-opms69faj">Benchmark</h2>We release <a target="_blank" rel="ugc" href="https://huggingface.co/datasets/stepfun-ai/GEdit-Bench">GEdit-Bench</a> as a new benchmark, grounded in real-world usages is developed to support more authentic and comprehensive evaluation. This benchmark, which is carefully curated to reflect actual user editing needs and a wide range of editing scenarios, enables more authentic and comprehensive evaluations of image editing models. Part results of the benchmark are shown below:<img src="https://huggingface.co/stepfun-ai/Step1X-Edit/resolve/main/assets/eval_res_en.png" alt="results" /><h2 id="citation-dyjh3jf9q">Citation</h2><pre><code>@article{liu2025step1x-edit,
 title={Step1X-Edit: A Practical Framework for General Image Editing}, 
 author={Shiyu Liu and Yucheng Han and Peng Xing and Fukun Yin and Rui Wang and Wei Cheng and Jiaqi Liao and Yingming Wang and Honghao Fu and Chunrui Han and Guopeng Li and Yuang Peng and Quan Sun and Jingwei Wu and Yan Cai and Zheng Ge and Ranchen Ming and Lei Xia and Xianfang Zeng and Yibo Zhu and Binxing Jiao and Xiangyu Zhang and Gang Yu and Daxin Jiang},
 journal={arXiv preprint arXiv:2504.17761},
 year={2025}
}</code></pre>

ComfyUI nodes-ACESTEP Music Generation & Step1X-Edit ComfyUI组件包及示例工作流

5.11 updated ACESTEP Audio2Audio workflowsACE-Step: A Step Towards Music Generation Foundation ModelACE Step 是一种新颖的开源音乐生成基础模型Model DescriptionACE-Step is a novel open-source foundation model for music generation that overcomes key limitations of existing approaches through a holistic architectural design. It integrates diffusion-based generation with Sana's Deep Compression AutoEncoder (DCAE) and a lightweight linear transformer, achieving state-of-the-art performance in generation speed, musical coherence, and controllability.Key Features:<ul><li>15× faster than LLM-based baselines (20s for 4-minute music on A100)</li><li>Superior musical coherence across melody, harmony, and rhythm</li><li>full-song generation, duration control and accepts natural language descriptions<a target="_blank" rel="ugc" href="https://github.com/ACE-Step/ACE-Step/raw/main/fig/ACE-Step_framework.png"> </a></li></ul>ACE Step是一种新颖的开源音乐生成基础模型，通过整体架构设计克服了现有方法的关键局限性。它将基于扩散的生成与Sana的深度压缩自动编码器（DCAE）和轻量级线性变压器集成在一起，在生成速度、音乐连贯性和可控性方面实现了最先进的性能。主要特点：比基于LLM的基线快15倍（A100上4分钟音乐为20秒）旋律、和声和节奏之间具有卓越的音乐连贯性完整的歌曲生成、持续时间控制和接受自然语言描述

Files with SHA256: abff262dcde7dddd14cc5b0bb071072251b0bbe39f5cea93110ca8e83921f3e6