A novel integrated approach has been proposed to enhance controlled architectural design rendering using diffusion models. The main goal of this study is to develop a hybrid diffusion model tailored to architectural design, which provides controlled, high-fidelity image generation for architectural elements while addressing limitations in existing generative models. Our method begins with a comparative analysis of various diffusion models, merging the top-performing models to create a hybrid ArchVisMix model. We employ ControlNet alongside this hybrid model as a base and optimize the parameters to achieve precise
control over architectural image generation. Additionally, we utilize the Flux model with ControlNet, which is particularly sensitive to parameter changes, allowing us to define a standardized set of parameters for generation. This ensures consistency and reliability in our results. Finally, we fine-tune our hybrid model using Low-Rank Adaptation (LoRA), allowing for more efficient training and better adaptation to architecture-specific prompts using a limited dataset. By integrating LoRA and ControlNet, our model not only overcomes the data limitation but also ensures more reliable and precise control over architectural outputs, allowing designers to generate high-quality, context-specific designs. This breakthrough has the potential to redefine how generative models are used in architectural practice, providing more tailored and effective tools for architects and designers.
Hybrid Diffusion Model: Developed a specialized diffusion model (ArchVisMix) by integrating top-performing generative models for architectural rendering.
ControlNet Integration: Utilized ControlNet to refine image generation, ensuring precise control over architectural elements.
Flux Model Optimization: Incorporated the Flux model with ControlNet to fine-tune parameter sensitivity for standardized image generation.
LoRA Fine-Tuning: Applied Low-Rank Adaptation (LoRA) to enhance model adaptability with limited training data.
Comparative Analysis: Conducted performance benchmarking of various diffusion models to select the most effective components.
Data-Efficient Training: Overcame data scarcity by optimizing generative capabilities with LoRA, ensuring high-quality outputs.
Context-Specific Design Generation: Enabled architects to generate controlled, high-fidelity designs tailored to specific requirements.
Generative AI for Architecture: Pioneered the use of hybrid diffusion models in architectural practice, enhancing design precision.
