Proteus v0.6

I'm excited to introduce Proteus v0.6, a complete rebuild of my AI image generation model. This is the first version of the rework, focusing entirely on enhancing photorealism. While it's not aiming to be state-of-the-art, I believe it's a good step forward in producing high-quality images. Please note that this is a preliminary version, and it's not the final, fully-featured checkpoint—more improvements and features will come in future updates.

Overview

Proteus v0.6 is a total rework from the ground up. In previous versions, combining different training methods and learning rates caused the model to become unstable during large-scale training. Learning from those experiences, I've retrained the model using only the photorealism aspects of the Proteus dataset.

For now, I'm calling this new training technique Multi-Perspective Fusion.

Multi-Perspective Fusion

This approach involves:

• Training Multiple LoRAs and Full-Parameter Checkpoints: I trained several Low-Rank Adaptation (LoRA) modules and full-parameter checkpoints on the same dataset multiple times to capture different "perspectives" of the data.

• Integrating into an Overarching Framework: These varied models are then combined within a larger framework to enhance overall performance.

I'm hoping this method will be interesting to data scientists exploring advanced training techniques.

Key Improvements in v0.6

• Total Rebuild: Constructed entirely from scratch to address previous issues.

• Enhanced Photorealism: Focused on producing good-quality photorealistic images.

• Stable Training Process: Refined training methods to prevent the model from falling apart during large-scale training.

• Preliminary Version: This is the first version of the rework; expect more features and improvements in future releases.

Limitations

• No Illustrations or Anime: Currently, the model can't generate illustrations or anime-style images because it's only been trained on photorealistic data.

• Not State-of-the-Art: While the model performs well, I'm not claiming it's state-of-the-art—just that it's a good starting point.

• Work in Progress: This is not the final, fully-featured checkpoint. More updates are planned.

Usage

Recommended Settings

• Clip Skip: 1

• CFG Scale: 7

• Steps: 25 - 50

• Sampler: DPM++ 2M SDE

• Scheduler: Karras

• Resolution: 1024x1024

Versions before v0.6

Proteus's Background

Proteus serves as a sophisticated enhancement over OpenDalleV1.1, leveraging its core functionalities to deliver superior outcomes. Key areas of advancement include heightened responsiveness to prompts and augmented creative capacities. To achieve this, it was fine-tuned using approximately 220,000 GPTV captioned images from copyright-free stock images (with some anime included), which were then normalized. Additionally, DPO (Direct Preference Optimization) was employed through a collection of 10,000 carefully selected high-quality, AI-generated image pairs. In pursuit of optimal performance, numerous LORA (Low-Rank Adaptation) models are trained independently before being selectively incorporated into the principal model via dynamic application methods. These techniques involve targeting particular segments within the model while avoiding interference with other areas during the learning phase. Consequently, Proteus exhibits marked improvements in portraying intricate facial characteristics and lifelike skin textures, all while sustaining commendable proficiency across various aesthetic domains, notably surrealism, anime, and cartoon-style visualizations.