Transforming E-Commerce Creativity: Strategic Deployment of Fine-Tuned Stability AI Models for PietraStudio Using AWS SageMaker.

Client

Pietra

Location

New York, NY

Industry

New York, NY

Services & Tech

Jupyter Notebook, Sagemaker, AWS Bedrock

Project Overview

In an era where creative and practical applications of AI are continuously expanding, Pietra, a trailblazer in e-commerce solutions, sought to harness the power of generative AI models to enhance their online store offerings. This initiative aimed to conduct a meticulous comparative analysis of the Stability Diffusion and DALL-E models using advanced techniques. The project’s cornerstone was the development on AWS Sagemaker to evaluate these models side-by-side on a uniform set of user-provided prompts. The primary goal was to explore and document each model’s capabilities, efficiency, and practical application potential in various creative scenarios.

The project entailed establishing a Sagemaker environment, fine-tuning the Stability Diffusion model on relevant datasets from e-commerce domain, generating and analyzing results from identical prompts, and comparing these with the results from the DALL-E model. The goal was to extract actionable insights to guide strategic model deployment in business operations.

The Problem

With the ever advancing Generative AI technology, there is a growing demand to leverage the best models; however, this brand-new creations come with challenges:

Model Performance Uncertainty: It is often unclear which generative model delivers superior performance as they are tested across a variety of prompts and use cases. This uncertainty complicates the selection process for specific applications.
Need for Fine-Tuning: There is a notable lack of insights on how fine-tuning techniques impact the performance of each model. This gap in knowledge makes it difficult to optimize models to meet specific needs effectively.
Complexity of Comparative Analysis: Establishing a systematic and fair comparison between the models presents significant difficulties.
Resource Constraints: The availability of computational resources for extensive model training/finetuning and evaluation is limited. This scarcity poses a substantial barrier to conducting thorough and continuous model improvements.
User Satisfaction: The ultimate measure of a model’s success is whether the generative outputs satisfy user expectations and fit the intended application requirements. Ensuring this alignment is crucial but challenging, given the subjective nature of user satisfaction.

Value of the Project

Being at the forefront is what allows a company to be the highlight of their domain; incurring into a barely explored field is nothing short of innovative. By leveraging and understanding which models and techniques are the most impactful to the use case, the project provided significant value to the client:

Performance Insights: Offered clear insights into the strengths and weaknesses of both Stability Diffusion and DALL-E models.
Resource Efficiency: Demonstrated efficient use of computational resources through optimized training and evaluation processes.
Enhanced Understanding: Improved the client’s understanding of generative AI capabilities and limitations.
Strategic Decision-Making: Enabled informed decision-making for future AI projects and model selection.
User Satisfaction: Ensured that the generated outputs aligned with user expectations and application needs.
Ease of replication: Provided detailed steps with easy to use instructions that allows the user’s technical team to replicate the best results without having to go through a discovery phase.

Solution

To present a feasible model, state of the art models (DALL-E and Stability Diffusion various models, mainly SDXL) were compared via Jupyter Runbooks, to allow developers to test different approaches:

Jupyter Environment Setup: Configured a Jupyter environment with necessary libraries and dependencies for both Stability Diffusion and DALL-E models.
Data Preparation: Used a clothes image captioning dataset from Hugging Face to customize data and outperform DALL-E outputs.

Model Fine-Tuning: To ease the comparison as well as the training of the models, the following architecture diagram was employed:

Architecture Diagram for training and inference. Having a defined a diagram helps the comparison task.

Applied advanced fine-tuning techniques to the Stability Diffusion model, including:

Textual Inversion
LoRA Fine Tuning (a GPU-efficient technique)
ControlNet for image-to-image tasks
Output Generation: Generated images and outputs for each prompt using both models.

LoRA model:

Text to image generation

Output :

Output :

Textual Inversion

Output :

Control Net

Output :

Result Analysis: Conducted a detailed analysis of the outputs, focusing on aspects such as quality, coherence, creativity, and computational efficiency

Generation with LoRA vs DALL-E: