Popular image creation AI Stable Diffusion needs text instructions to create an image. When given a vague prompt, it occasionally generates the exact result you want. Other times, you receive less-than-ideal results. Here are some pointers and ideas for getting the best outcomes.
How to write a prompt for Stable Diffusion
A well-written prompt is essential, as anyone who has used AI image generators like Stable Diffusion, DALL-E, or MidJourney will attest to. A well-written prompt can make the difference between getting a fantastic representation of your idea or getting some monstrosity from the uncanny valley with way too many fingers staring back at you.
Since Stable Diffusion launched in August 2022, people have been figuring out the best ways to achieve ideal results. And as the Stable Diffusion's checkpoints will frequently receive updates, finding the ideal prompt will be a moving target.
However, certain patterns have emerged that transform your imagination from meh to wow. So, let's see how we can up your game of prompting a little bit by learning the anatomy of prompts. The hierarchy that worked for me followed this pattern.
(A) Framing + (B) Medium + (C) Subject + (D) Style + (E) Extras
Sometimes I change the order, but almost always I use all those ingredients.
For example
A medium shot photo of a cute little labrador, polaroid, bokeh
where "A medium shot" is framing, "photo" is medium, "a cute little labrador" is subject, "Polaroid" is style, and "bokeh" is a medium specific effect.
So when I start imagining a new image, I usually follow this pattern to get great results. Now let's take a look at the specifics of each step.
Define your art medium first
As Stable Diffusion allows you to imagine almost anything, it is essential to start with the art medium you are interested in. It can be
- a photo,
- a painting,
- a sculpture,
- a 3D rendering, etc.
If you don't define it Stable Diffusion will provide some random medium for you.
Set the right "framing"
I didn't find a better term for the next step than "framing." Essentially, I try to define where the subject is positioned in my image. In case of photos, for example, I use tags like "long shot," "medium shot," and "close up" to indicate where the subject will be in my image. In the case of paintings, I have used tags like "landscape," "portrait," etc.
For example: "A landscape painting of the dark forest."
Define your subject as specifically as possible.
Although I have the subject in mind before I start typing anything else into the prompt, I usually add it in the second or third position.
Stable Diffusion likes very specific prompts. Especially when compared to MidJourney. You must be very clear about what you want out of it.
So instead of typing in "a dog", try typing in "a cute little labrador with blue eyes and golden fur" and you may get something like this.
Define the style
The need for specificity remains even after you've fully described the desired image's subject matter. You can (and should) also specify the preferred style.
Generally speaking, it's best to start with the fewest number of keywords to get close to your goal before adding more to hone in on a specific aesthetic
If you want to, Stable Diffusion can imitate particular artists in addition to imitating particular styles or media.
You should definitely try throwing "by (artist name)" into your prompts if you haven’t already. It tends to produce dramatic results. Don’t be afraid to combine artists that wouldn’t normally go together, either. Oftentimes, Stable Diffusion will combine their respective styles in exciting and unpredictable ways.
For example, you can try "A watercolor painting, of a bucket of red roses on a table by Monet" produces quite Monet-like painting.
Extras
Finally, you can enhance your images further. You can add extra modifier tags like "bokeh," "tilt shift," or "long exposure" to your prompts for photos to give them more flair. If you are more into digital paintings or 3D rendering, style tags like "unreal engine," "octane rendering," "8K," and "trending in Artstation" can definitely bump up your final result.
For example: "a painting of a cute and adorable little royal labrador in forest, unreal engine, atmospheric lighting, octane rendering, 8k, trending in Artstation, by Jordan Nelson and pixelkitties" will return something like that.
Pro-tip: Weigh Your Keywords
Of course, simply including keywords in your prompt will only get you so far. What if you include all of the necessary elements in your prompt but they are not properly proportioned?
Stable Diffusion supports weighting of prompt keywords. In other words, you can tell it that it really needs to pay attention to a specific keyword (or keywords) and pay less attention to others. It is handy if you’re getting results that are kinda what you’re looking for, but not quite there.
In the command-line version of Stable Diffusion, you just add a full colon followed by a decimal number to the word you want to emphasize.
Most of the GUIs available for Stable Diffusion handle weighting without requiring you to enter explicit percentages. You can usually add parenthesis (or many) around a term in your prompt to emphasize it, and brackets to decrease the importance of a term.
To draw attention to the red ball in your image, try using brackets and parentheses like this: "a photo of a [cute little labrador] with a ((red ball))"
Learn from the best and experiment
The popularity of Stable Diffusion and other AI art generators has skyrocketed. This type of AI art can be found all over the place. That means there are now millions of user-generated images floating around the internet, and most of the time, people include the prompt they used to get their results.
Here are a few resources to help spark your inspiration if you aren’t sure what you’re looking to create:
- Lexica: a collection of images created with Stable Diffusion and the corresponding prompt Keyword searchable.
- Stable Diffusion Artist Style Studies: A non-exhaustive list of artists Stable Diffusion may be familiar with, along with general descriptions of their artistic style.
- Stable Diffusion Modifier Studies: a list of modifiers that can be used in your "extras" section of the prompt.
- The AI Art Modifiers List: A photo gallery displaying some of the most powerful modifiers you can use in your prompts, as well as what they do. They are organized by modifier type.
One thing to remember is that the output in the case of Stable Diffusion is always a moving target. No matter how detailed your prompt is, there will be no two identical images. Therefore, it is always a good idea to generate multiple images from the same prompt and then choose the one you like best. Happy experimenting!
