Outpainting and outcropping#
Outpainting is a process by which the AI generates parts of the image that are outside its original frame. It can be used to fix up images in which the subject is off center, or when some detail (often the top of someone's head!) is cut off.
InvokeAI supports two versions of outpainting, one called "outpaint" and the other "outcrop." They work slightly differently and each has its advantages and drawbacks.
Outpainting is the same as inpainting, except that the painting occurs in the
regions outside of the original image. To outpaint using the
line script, prepare an image in which the borders to be extended are pure
black. Add an alpha channel (if there isn't one already), and make the borders
completely transparent and the interior completely opaque. If you wish to modify
the interior as well, you may create transparent holes in the transparency
img2img will paint into as usual.
Pass the image as the argument to the
-I switch as you would for regular
You'll likely be delighted by the results.
Do not try to expand the image too much at once. Generally it is best to expand the margins in 64-pixel increments. 128 pixels often works, but your mileage may vary depending on the nature of the image you are trying to outpaint into.
There are a series of switches that can be used to adjust how the inpainting algorithm operates. In particular, you can use these to minimize the seam that sometimes appears between the original image and the extended part. These switches are:
||Size of the mask around the seam between original and outpainted image|
||The amount to blur the seam inwards|
||The img2img strength to use when filling the seam|
||The number of steps to use to fill the seam.|
||The tile size to use for filling outpaint areas|
outcrop extension gives you a convenient
!fix postprocessing command
that allows you to extend a previously-generated image in 64 pixel increments in
any direction. You can apply the module to any image previously-generated by
InvokeAI. Note that it works with arbitrary PNG photographs, but not currently
with JPG or other formats. Outcropping is particularly effective when combined
runwayML custom inpainting model.
Consider this image:
Pretty nice, but it's annoying that the top of her head is cut off. She's also a bit off center. Let's fix that!
This is saying to apply the
outcrop extension by extending the top of the
image by 128 pixels, and the right and bottom of the image by 64 pixels. You can
use any combination of top|left|right|bottom, and specify any number of pixels
to extend. You can also abbreviate
The result looks like this:
The new image is larger than the original (576x704) because 64 pixels were added to the top and right sides. You will need enough VRAM to process an image of this size.
Outcropping non-InvokeAI images#
You can outcrop an arbitrary image that was not generated by InvokeAI,
but your results will vary. The
inpainting-1.5 model is highly
recommended, but if not feasible, then you may be able to improve the
output by conditioning the outcropping with a text prompt that
describes the scene using the
You may also provide a different seed for outcropping to use by passing
-S<seed>. A negative seed will generate a new random seed.
A number of caveats:
Although you can specify any pixel values, they will be rounded up to the nearest multiple of 64. Smaller values are better. Larger extensions are more likely to generate artefacts. However, if you wish you can run the !fix command repeatedly to cautiously expand the image.
The extension is stochastic, meaning that each time you run it you'll get a slightly different result. You can run it repeatedly until you get an image you like. Unfortunately
!fixdoes not currently respect the
Your results will be much better if you use the
inpaint-1.5model released by runwayML and installed by default by
invokeai-configure. This model was trained specifically to harmoniously fill in image gaps. The standard model will work as well, but you may notice color discontinuities at the border.
When using the
inpaint-1.5model, you may notice subtle changes to the area outside the masked region. This is because the model performs an encoding/decoding on the image as a whole. This does not occur with the standard model.
outpaint extension does the same thing, but with subtle differences.
Starting with the same image, here is how we would add an additional 64 pixels
to the top of the image:
(you can abbreviate
The result is shown here:
Although the effect is similar, there are significant differences from outcropping:
- You can only specify one direction to extend at a time.
- The image is not resized. Instead, the image is shifted by the specified number of pixels. If you look carefully, you'll see that less of the lady's torso is visible in the image.
- Because the image dimensions remain the same, there's no rounding to multiples of 64.
- Attempting to outpaint larger areas will frequently give rise to ugly ghosting effects.
- For best results, try increasing the step number.
- If you don't specify a pixel value in
-D, it will default to half of the whole image, which is likely not what you want.
outcrop are perfect, but we continue to tune
and improve them. If one doesn't work, try the other. You may also
wish to experiment with other
img2img arguments, such as
Created: September 21, 2022