Skip to content

Invocation API#

Each invocation's invoke method is provided a single arg - the Invocation Context.

This object provides an API the invocation can use to interact with application services, for example:

  • Saving images
  • Logging messages
  • Loading models
class MyInvocation(BaseInvocation):
  ...
  def invoke(self, context: InvocationContext) -> ImageOutput:
      # Load an image
      image_pil = context.images.get_pil(self.image.image_name)
      # Do something to the image
      output_image = do_something_cool(image_pil)
      # Save the image
      image_dto = context.images.save(output_image)
      # Log a message
      context.logger.info(f"Did something cool, image saved!")
      # Return the output
      return ImageOutput.build(image_dto)
      ...

The full API is documented below.

Mixins#

Two important mixins are provided to facilitate working with metadata and gallery boards.

WithMetadata#

Inherit from this class (in addition to BaseInvocation) to add a metadata input to your node. When you do this, you can access the metadata dict from self.metadata in the invoke() function.

The dict will be populated via the node's input, and you can add any metadata you'd like to it. When you call context.images.save(), if the metadata dict has any data, it be automatically embedded in the image.

WithBoard#

Inherit from this class (in addition to BaseInvocation) to add a board input to your node. This renders as a drop-down to select a board. The user's selection will be accessible from self.board in the invoke() function.

When you call context.images.save(), if a board was selected, the image will added to that board as it is saved.

InvocationContext #

Provides access to various services and data for the current invocation.

Attributes:

Name Type Description
images ImagesInterface

Methods to save, get and update images and their metadata.

tensors TensorsInterface

Methods to save and get tensors, including image, noise, masks, and masked images.

conditioning ConditioningInterface

Methods to save and get conditioning data.

models ModelsInterface

Methods to check if a model exists, get a model, and get a model's info.

logger LoggerInterface

The app logger.

config ConfigInterface

The app config.

util UtilInterface

Utility methods, including a method to check if an invocation was canceled and step callbacks.

boards BoardsInterface

Methods to interact with boards.

ImagesInterface #

get_dto #

get_dto(image_name: str) -> ImageDTO

Gets an image as an ImageDTO object.

Parameters:

Name Type Description Default
image_name str

The name of the image to get.

required

Returns:

Type Description
ImageDTO

The image as an ImageDTO object.

get_metadata #

get_metadata(image_name: str) -> Optional[MetadataField]

Gets an image's metadata, if it has any.

Parameters:

Name Type Description Default
image_name str

The name of the image to get the metadata for.

required

Returns:

Type Description
Optional[MetadataField]

The image's metadata, if it has any.

get_path #

get_path(image_name: str, thumbnail: bool = False) -> Path

Gets the internal path to an image or thumbnail.

Parameters:

Name Type Description Default
image_name str

The name of the image to get the path of.

required
thumbnail bool

Get the path of the thumbnail instead of the full image

False

Returns:

Type Description
Path

The local path of the image or thumbnail.

get_pil #

get_pil(image_name: str, mode: IMAGE_MODES | None = None) -> Image

Gets an image as a PIL Image object.

Parameters:

Name Type Description Default
image_name str

The name of the image to get.

required
mode IMAGE_MODES | None

The color mode to convert the image to. If None, the original mode is used.

None

Returns:

Type Description
Image

The image as a PIL Image object.

save #

save(image: Image, board_id: Optional[str] = None, image_category: ImageCategory = ImageCategory.GENERAL, metadata: Optional[MetadataField] = None) -> ImageDTO

Saves an image, returning its DTO.

If the current queue item has a workflow or metadata, it is automatically saved with the image.

Parameters:

Name Type Description Default
image Image

The image to save, as a PIL image.

required
board_id Optional[str]

The board ID to add the image to, if it should be added. It the invocation inherits from WithBoard, that board will be used automatically. Use this only if you want to override or provide a board manually!

None
image_category ImageCategory

The category of the image. Only the GENERAL category is added to the gallery.

GENERAL
metadata Optional[MetadataField]

The metadata to save with the image, if it should have any. If the invocation inherits from WithMetadata, that metadata will be used automatically. Use this only if you want to override or provide metadata manually!

None

Returns:

Type Description
ImageDTO

The saved image DTO.

TensorsInterface #

load #

load(name: str) -> Tensor

Loads a tensor by name.

Parameters:

Name Type Description Default
name str

The name of the tensor to load.

required

Returns:

Type Description
Tensor

The loaded tensor.

save #

save(tensor: Tensor) -> str

Saves a tensor, returning its name.

Parameters:

Name Type Description Default
tensor Tensor

The tensor to save.

required

Returns:

Type Description
str

The name of the saved tensor.

ConditioningInterface #

load #

load(name: str) -> ConditioningFieldData

Loads conditioning data by name.

Parameters:

Name Type Description Default
name str

The name of the conditioning data to load.

required

Returns:

Type Description
ConditioningFieldData

The loaded conditioning data.

save #

save(conditioning_data: ConditioningFieldData) -> str

Saves a conditioning data object, returning its name.

Parameters:

Name Type Description Default
conditioning_data ConditioningFieldData

The conditioning data to save.

required

Returns:

Type Description
str

The name of the saved conditioning data.

ModelsInterface #

Common API for loading, downloading and managing models.

download_and_cache_model #

download_and_cache_model(source: str | AnyHttpUrl) -> Path

Download the model file located at source to the models cache and return its Path.

This can be used to single-file install models and other resources of arbitrary types which should not get registered with the database. If the model is already installed, the cached path will be returned. Otherwise it will be downloaded.

Parameters:

Name Type Description Default
source str | AnyHttpUrl

A URL that points to the model, or a huggingface repo_id.

required

Returns:

Type Description
Path

Path to the downloaded model

exists #

exists(identifier: Union[str, ModelIdentifierField]) -> bool

Check if a model exists.

Parameters:

Name Type Description Default
identifier Union[str, ModelIdentifierField]

The key or ModelField representing the model.

required

Returns:

Type Description
bool

True if the model exists, False if not.

get_config #

get_config(identifier: Union[str, ModelIdentifierField]) -> AnyModelConfig

Get a model's config.

Parameters:

Name Type Description Default
identifier Union[str, ModelIdentifierField]

The key or ModelField representing the model.

required

Returns:

Type Description
AnyModelConfig

The model's config.

load #

load(identifier: Union[str, ModelIdentifierField], submodel_type: Optional[SubModelType] = None) -> LoadedModel

Load a model.

Parameters:

Name Type Description Default
identifier Union[str, ModelIdentifierField]

The key or ModelField representing the model.

required
submodel_type Optional[SubModelType]

The submodel of the model to get.

None

Returns:

Type Description
LoadedModel

An object representing the loaded model.

load_by_attrs #

load_by_attrs(name: str, base: BaseModelType, type: ModelType, submodel_type: Optional[SubModelType] = None) -> LoadedModel

Load a model by its attributes.

Parameters:

Name Type Description Default
name str

Name of the model.

required
base BaseModelType

The models' base type, e.g. BaseModelType.StableDiffusion1, BaseModelType.StableDiffusionXL, etc.

required
type ModelType

Type of the model, e.g. ModelType.Main, ModelType.Vae, etc.

required
submodel_type Optional[SubModelType]

The type of submodel to load, e.g. SubModelType.UNet, SubModelType.TextEncoder, etc. Only main

None

Returns:

Type Description
LoadedModel

An object representing the loaded model.

load_local_model #

load_local_model(model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None) -> LoadedModelWithoutConfig

Load the model file located at the indicated path

If a loader callable is provided, it will be invoked to load the model. Otherwise, safetensors.torch.load_file() or torch.load() will be called to load the model.

Be aware that the LoadedModelWithoutConfig object has no config attribute

Parameters:

Name Type Description Default
path

A model Path

required
loader Optional[Callable[[Path], AnyModel]]

A Callable that expects a Path and returns a dict[str|int, Any]

None

Returns:

Type Description
LoadedModelWithoutConfig

A LoadedModelWithoutConfig object.

load_remote_model #

load_remote_model(source: str | AnyHttpUrl, loader: Optional[Callable[[Path], AnyModel]] = None) -> LoadedModelWithoutConfig

Download, cache, and load the model file located at the indicated URL or repo_id.

If the model is already downloaded, it will be loaded from the cache.

If the a loader callable is provided, it will be invoked to load the model. Otherwise, safetensors.torch.load_file() or torch.load() will be called to load the model.

Be aware that the LoadedModelWithoutConfig object has no config attribute

Parameters:

Name Type Description Default
source str | AnyHttpUrl

A URL or huggingface repoid.

required
loader Optional[Callable[[Path], AnyModel]]

A Callable that expects a Path and returns a dict[str|int, Any]

None

Returns:

Type Description
LoadedModelWithoutConfig

A LoadedModelWithoutConfig object.

search_by_attrs #

search_by_attrs(name: Optional[str] = None, base: Optional[BaseModelType] = None, type: Optional[ModelType] = None, format: Optional[ModelFormat] = None) -> list[AnyModelConfig]

Search for models by attributes.

Parameters:

Name Type Description Default
name Optional[str]

The name to search for (exact match).

None
base Optional[BaseModelType]

The base to search for, e.g. BaseModelType.StableDiffusion1, BaseModelType.StableDiffusionXL, etc.

None
type Optional[ModelType]

Type type of model to search for, e.g. ModelType.Main, ModelType.Vae, etc.

None
format Optional[ModelFormat]

The format of model to search for, e.g. ModelFormat.Checkpoint, ModelFormat.Diffusers, etc.

None

Returns:

Type Description
list[AnyModelConfig]

A list of models that match the attributes.

search_by_path #

search_by_path(path: Path) -> list[AnyModelConfig]

Search for models by path.

Parameters:

Name Type Description Default
path Path

The path to search for.

required

Returns:

Type Description
list[AnyModelConfig]

A list of models that match the path.

LoggerInterface #

debug #

debug(message: str) -> None

Logs a debug message.

Parameters:

Name Type Description Default
message str

The message to log.

required

error #

error(message: str) -> None

Logs an error message.

Parameters:

Name Type Description Default
message str

The message to log.

required

info #

info(message: str) -> None

Logs an info message.

Parameters:

Name Type Description Default
message str

The message to log.

required

warning #

warning(message: str) -> None

Logs a warning message.

Parameters:

Name Type Description Default
message str

The message to log.

required

ConfigInterface #

get #

Gets the app's config.

Returns:

Type Description
InvokeAIAppConfig

The app's config.

UtilInterface #

flux_step_callback #

flux_step_callback(intermediate_state: PipelineIntermediateState) -> None

The step callback emits a progress event with the current step, the total number of steps, a preview image, and some other internal metadata.

This should be called after each denoising step.

Parameters:

Name Type Description Default
intermediate_state PipelineIntermediateState

The intermediate state of the diffusion pipeline.

required

is_canceled #

is_canceled() -> bool

Checks if the current session has been canceled.

Returns:

Type Description
bool

True if the current session has been canceled, False if not.

sd_step_callback #

sd_step_callback(intermediate_state: PipelineIntermediateState, base_model: BaseModelType) -> None

The step callback emits a progress event with the current step, the total number of steps, a preview image, and some other internal metadata.

This should be called after each denoising step.

Parameters:

Name Type Description Default
intermediate_state PipelineIntermediateState

The intermediate state of the diffusion pipeline.

required
base_model BaseModelType

The base model for the current denoising step.

required

signal_progress #

signal_progress(message: str, percentage: float | None = None, image: Image | None = None, image_size: tuple[int, int] | None = None) -> None

Signals the progress of some long-running invocation. The progress is displayed in the UI.

If a percentage is provided, the UI will display a progress bar and automatically append the percentage to the message. You should not include the percentage in the message.

Example
total_steps = 10
for i in range(total_steps):
    percentage = i / (total_steps - 1)
    context.util.signal_progress("Doing something cool", percentage)

If an image is provided, the UI will display it. If your image should be displayed at a different size, provide a tuple of (width, height) for the image_size parameter. The image will be displayed at the specified size in the UI.

For example, SD denoising progress images are ⅛ the size of the original image, so you'd do this to ensure the image is displayed at the correct size:

# Calculate the output size of the image (8x the progress image's size)
width = progress_image.width * 8
height = progress_image.height * 8
# Signal the progress with the image and output size
signal_progress("Denoising", percentage, progress_image, (width, height))

If your progress image is very large, consider downscaling it to reduce the payload size and provide the original size to the image_size parameter. The PIL thumbnail method is useful for this, as it maintains the aspect ratio of the image:

# `thumbnail` modifies the image in-place, so we need to first make a copy
thumbnail_image = progress_image.copy()
# Resize the image to a maximum of 256x256 pixels, maintaining the aspect ratio
thumbnail_image.thumbnail((256, 256))
# Signal the progress with the thumbnail, passing the original size
signal_progress("Denoising", percentage, thumbnail, progress_image.size)

Parameters:

Name Type Description Default
message str

A message describing the current status. Do not include the percentage in this message.

required
percentage float | None

The current percentage completion for the process. Omit for indeterminate progress.

None
image Image | None

An optional image to display.

None
image_size tuple[int, int] | None

The optional size of the image to display. If omitted, the image will be displayed at its original size.

None

BoardsInterface #

add_image_to_board #

add_image_to_board(board_id: str, image_name: str) -> None

Adds an image to a board.

Parameters:

Name Type Description Default
board_id str

The ID of the board to add the image to.

required
image_name str

The name of the image to add to the board.

required

create #

create(board_name: str) -> BoardDTO

Creates a board.

Parameters:

Name Type Description Default
board_name str

The name of the board to create.

required

Returns:

Type Description
BoardDTO

The created board DTO.

get_all #

get_all() -> list[BoardDTO]

Gets all boards.

Returns:

Type Description
list[BoardDTO]

A list of all boards.

get_all_image_names_for_board #

get_all_image_names_for_board(board_id: str) -> list[str]

Gets all image names for a board.

Parameters:

Name Type Description Default
board_id str

The ID of the board to get the image names for.

required

Returns:

Type Description
list[str]

A list of all image names for the board.

get_dto #

get_dto(board_id: str) -> BoardDTO

Gets a board DTO.

Parameters:

Name Type Description Default
board_id str

The ID of the board to get.

required

Returns:

Type Description
BoardDTO

The board DTO.