Creating Celebrity Look-Alike Images with Stable Diffusion

Introduction

We all have idols or people we admire, figures from movies, music, sports, or history who have left a lasting impression on us. Naturally, we enjoy seeing familiar faces in the work we create, whether for artistic inspiration, storytelling, or just for fun. With AI tools like Stable Diffusion, it’s now possible to generate images that resemble well-known individuals.

Stable Diffusion has become one of the most powerful AI models for generating photorealistic images, allowing users to create portraits that bear a striking resemblance to real celebrities. However, while this technology is impressive, it also raises ethical and legal concerns. In this guide, we’ll explore how Stable Diffusion can be used to generate celebrity look-alike images, the best techniques to achieve high-quality results, and the responsible use of this technology.

Understanding Stable Diffusion’s Capabilities

Stable Diffusion is a text-to-image AI model that can generate detailed images based on prompts. While it doesn’t directly recreate real celebrities, users can fine-tune prompts and apply specific techniques to achieve results that resemble well-known individuals.

If you are not familiar with how to use Stable Diffusion, we recommend checking out a few guides listed below:

• Using Stable Diffusion WebUI Docker for Image Generation – A Newbie's Guide
• Stable Diffusion Parameter Guide
• Mastering Human Portraits

Key Features of Stable Diffusion for This Task

• Prompt Control – Through careful prompt control (prompt scheduling and prompt weighting), you can achieve the desired effect and likeness.
• Fine-Tuning – Using pre-trained models that enhance facial similarities.

Prompt Control

Some Warm Ups

Through some testing, we can see how well Stable Diffusion works in generating images that resemble well-known individuals.

Attempt #1

Abraham Lincoln

Attempt #2

Parameters:

• Steps: 20
• Sampler: DPM++ 2M
• Schedule type: Karras
• CFG scale: 12
• Seed: 4221999450
• Size: 512x512
• Model hash: c6bbc15e32
• Model: sd-v1-5-inpainting
• Conditional mask weight: 1.0
• Version: v1.9.4

Now adding more details:

A highly detailed, colorful, photorealistic portrait of Abraham Lincoln, 4K resolution, dramatic lighting, sharp facial details, deep wrinkles, strong jawline, piercing eyes, dignified expression, black suit with bow tie, cinematic composition, DSLR shot, volumetric lighting, ultra-HD, realistic skin texture, ultra quality, dof, film grain, Fujifilm XT3, crystal clear

Negative prompt:

deformed face, disfigured, asymmetric, duplicate, tattoos, marks, Extra heads, extra faces, multiple faces, extra eyes, immature, cartoon, anime, 3d, painting, b&w, multiple figures, multiple heads, extra hands

Stable Diffusion does a great job of recreating the likeness with a more detailed prompts.

Prompt Scheduling and Weighting

Now we are gonna play with prompt scheduling and weighting to achieve the desired effect.

Prompt Scheduling:

Prompt scheduling in Stable Diffusion refers to the technique of modifying the text prompt dynamically over the course of the image generation process. Instead of using a single static prompt, prompt scheduling allows for gradual changes in the description, influencing how the model interprets and renders the image over multiple denoising steps. With this technique, we can precisely control the generated person's face and seamlessly blend the features of two celebrities.

Prompt scheduling is typically available in automatic1111’s WebUI, for example:

A photorealistic portrait of a woman | A Van Gogh-style painting of the same woman :0.5

The :0.5 ratio means the prompt starts shifting at 50% of the total steps.

Now back to the task at hand, we change the prompt to:

A highly detailed, colorful, photorealistic portrait of Abraham Lincoln | Henry Cavill :0.5, 4K resolution, dramatic lighting, sharp facial details, deep wrinkles, strong jawline, piercing eyes, dignified expression, black suit with bow tie, cinematic composition, DSLR shot, volumetric lighting, ultra-HD, realistic skin texture, ultra quality, dof, film grain, Fujifilm XT3, crystal clear

Here are our Cavilcoln:

Prompt Weighting:

Prompt weighting in Stable Diffusion allows users to assign different levels of importance to specific words or phrases in a prompt. This helps the AI model prioritize certain details over others when generating an image.

By adjusting the weight of words, you can fine-tune the composition, style, or specific elements of the image without needing extensive prompt rewriting.

Usage:

In prompt, parentheses () and brackets [] are used for weighting:

• Parentheses () increase emphasis on a word or phrase.
• Brackets [] decrease emphasis, making an element less prominent.

Now we change the prompt to:

A highly detailed, photorealistic image of Abraham Lincoln | Henry Cavill :0.6 sitting on a (chair:2.3), black suit with bow tie, [history accuracy] , 4K resolution, cinematic lighting, DSLR shot, ultra-HD, realistic skin texture, cinematic composition, DSLR shot, volumetric lighting, ultra-HD, realistic skin texture, ultra quality, dof, film grain, Canon 70D

Parameters: Steps: 20, Sampler: DPM2, Schedule type: Karras, CFG scale: 10, Seed: 1101313381, Size: 512x512, Model hash: c6bbc15e32, Model: sd-v1-5-inpainting, Conditional mask weight: 1.0, Version: v1.9.4

Using Pre-trained Models for Better Results

There are models that have been tuned specifically for creating human portrait images. A list of those models are:

• F222: Known for better face generation.
• RealisticVision: A popular choice for photorealistic AI art.
• ChilloutMix: Good for anime-style and realistic portraits, especially for Asian facial features.

For the details of how to use these models, please check out this article.

Now we test the prompts that have been used previously. Below you can see the difference between Stable Diffusion base model and the models mentioned above.

With model F222:

Final Thoughts

While AI-generated images can be used for artistic and creative purposes, it is crucial to avoid generating images that impersonate real people for deceptive, defamatory, or commercial exploitation. And remember, these techniques have the potential to be used for unauthorized purposes, so consider the legal and ethical implications of your work.

Stable Diffusion offers an exciting way to explore AI-generated art, but ethical responsibility is key. By focusing on creativity while respecting privacy and legal boundaries, users can harness this technology without engaging in harmful practices.

References

• How to generate realistic people in Stable Diffusion

Get in touch

For further information, I can be reach via:

• X (formally twitter): Eric Tang
• email: [email protected]