Invocation API#

Each invocation's invoke method is provided a single arg - the Invocation Context.

This object provides an API the invocation can use to interact with application services, for example:

Saving images
Logging messages
Loading models

class MyInvocation(BaseInvocation):
  ...
  def invoke(self, context: InvocationContext) -> ImageOutput:
      # Load an image
      image_pil = context.images.get_pil(self.image.image_name)
      # Do something to the image
      output_image = do_something_cool(image_pil)
      # Save the image
      image_dto = context.images.save(output_image)
      # Log a message
      context.logger.info(f"Did something cool, image saved!")
      # Return the output
      return ImageOutput.build(image_dto)
      ...

The full API is documented below.

Mixins#

Two important mixins are provided to facilitate working with metadata and gallery boards.

`WithMetadata`#

Inherit from this class (in addition to BaseInvocation) to add a metadata input to your node. When you do this, you can access the metadata dict from self.metadata in the invoke() function.

The dict will be populated via the node's input, and you can add any metadata you'd like to it. When you call context.images.save(), if the metadata dict has any data, it be automatically embedded in the image.

`WithBoard`#

Inherit from this class (in addition to BaseInvocation) to add a board input to your node. This renders as a drop-down to select a board. The user's selection will be accessible from self.board in the invoke() function.

When you call context.images.save(), if a board was selected, the image will added to that board as it is saved.

InvocationContext #

Provides access to various services and data for the current invocation.

Attributes:

Name	Type	Description
`images`	`ImagesInterface`	Methods to save, get and update images and their metadata.
`tensors`	`TensorsInterface`	Methods to save and get tensors, including image, noise, masks, and masked images.
`conditioning`	`ConditioningInterface`	Methods to save and get conditioning data.
`models`	`ModelsInterface`	Methods to check if a model exists, get a model, and get a model's info.
`logger`	`LoggerInterface`	The app logger.
`config`	`ConfigInterface`	The app config.
`util`	`UtilInterface`	Utility methods, including a method to check if an invocation was canceled and step callbacks.
`boards`	`BoardsInterface`	Methods to interact with boards.

ImagesInterface #

Methods:

Name	Description
`get_dto`	Gets an image as an ImageDTO object.
`get_metadata`	Gets an image's metadata, if it has any.
`get_path`	Gets the internal path to an image or thumbnail.
`get_pil`	Gets an image as a PIL Image object. This method returns a copy of the image.
`save`	Saves an image, returning its DTO.

get_dto #

get_dto(image_name: str) -> ImageDTO

Gets an image as an ImageDTO object.

Parameters:

Name	Type	Description	Default
`image_name`	`str`	The name of the image to get.	required

Returns:

Type	Description
`ImageDTO`	The image as an ImageDTO object.

get_metadata #

get_metadata(image_name: str) -> Optional[MetadataField]

Gets an image's metadata, if it has any.

Parameters:

Name	Type	Description	Default
`image_name`	`str`	The name of the image to get the metadata for.	required

Returns:

Type	Description
`Optional[MetadataField]`	The image's metadata, if it has any.

get_path #

get_path(image_name: str, thumbnail: bool = False) -> Path

Gets the internal path to an image or thumbnail.

Parameters:

Name	Type	Description	Default
`image_name`	`str`	The name of the image to get the path of.	required
`thumbnail`	`bool`	Get the path of the thumbnail instead of the full image	`False`

Returns:

Type	Description
`Path`	The local path of the image or thumbnail.

get_pil #

get_pil(image_name: str, mode: IMAGE_MODES | None = None) -> Image

Gets an image as a PIL Image object. This method returns a copy of the image.

Parameters:

Name	Type	Description	Default
`image_name`	`str`	The name of the image to get.	required
`mode`	`IMAGE_MODES \| None`	The color mode to convert the image to. If None, the original mode is used.	`None`

Returns:

Type	Description
`Image`	The image as a PIL Image object.

save #

save(image: Image, board_id: Optional[str] = None, image_category: ImageCategory = GENERAL, metadata: Optional[MetadataField] = None) -> ImageDTO

Saves an image, returning its DTO.

If the current queue item has a workflow or metadata, it is automatically saved with the image.

Parameters:

Name	Type	Description	Default
`image`	`Image`	The image to save, as a PIL image.	required
`board_id`	`Optional[str]`	The board ID to add the image to, if it should be added. It the invocation inherits from `WithBoard`, that board will be used automatically. Use this only if you want to override or provide a board manually!	`None`
`image_category`	`ImageCategory`	The category of the image. Only the GENERAL category is added to the gallery.	`GENERAL`
`metadata`	`Optional[MetadataField]`	The metadata to save with the image, if it should have any. If the invocation inherits from `WithMetadata`, that metadata will be used automatically. Use this only if you want to override or provide metadata manually!	`None`

Returns:

Type	Description
`ImageDTO`	The saved image DTO.

TensorsInterface #

Methods:

Name	Description
`load`	Loads a tensor by name. This method returns a copy of the tensor.
`save`	Saves a tensor, returning its name.

load #

load(name: str) -> Tensor

Loads a tensor by name. This method returns a copy of the tensor.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the tensor to load.	required

Returns:

Type	Description
`Tensor`	The tensor.

save #

save(tensor: Tensor) -> str

Saves a tensor, returning its name.

Parameters:

Name	Type	Description	Default
`tensor`	`Tensor`	The tensor to save.	required

Returns:

Type	Description
`str`	The name of the saved tensor.

ConditioningInterface #

Methods:

Name	Description
`load`	Loads conditioning data by name. This method returns a copy of the conditioning data.
`save`	Saves a conditioning data object, returning its name.

load #

load(name: str) -> ConditioningFieldData

Loads conditioning data by name. This method returns a copy of the conditioning data.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the conditioning data to load.	required

Returns:

Type	Description
`ConditioningFieldData`	The conditioning data.

save #

save(conditioning_data: ConditioningFieldData) -> str

Saves a conditioning data object, returning its name.

Parameters:

Name	Type	Description	Default
`conditioning_data`	`ConditioningFieldData`	The conditioning data to save.	required

Returns:

Type	Description
`str`	The name of the saved conditioning data.

ModelsInterface #

Common API for loading, downloading and managing models.

Methods:

Name	Description
`download_and_cache_model`	Download the model file located at source to the models cache and return its Path.
`exists`	Check if a model exists.
`get_config`	Get a model's config.
`load`	Load a model.
`load_by_attrs`	Load a model by its attributes.
`load_local_model`	Load the model file located at the indicated path
`load_remote_model`	Download, cache, and load the model file located at the indicated URL or repo_id.
`search_by_attrs`	Search for models by attributes.
`search_by_path`	Search for models by path.

download_and_cache_model #

download_and_cache_model(source: str | AnyHttpUrl) -> Path

Download the model file located at source to the models cache and return its Path.

This can be used to single-file install models and other resources of arbitrary types which should not get registered with the database. If the model is already installed, the cached path will be returned. Otherwise it will be downloaded.

Parameters:

Name	Type	Description	Default
`source`	`str \| AnyHttpUrl`	A URL that points to the model, or a huggingface repo_id.	required

Returns:

Type	Description
`Path`	Path to the downloaded model

exists #

exists(identifier: Union[str, ModelIdentifierField]) -> bool

Check if a model exists.

Parameters:

Name	Type	Description	Default
`identifier`	`Union[str, ModelIdentifierField]`	The key or ModelField representing the model.	required

Returns:

Type	Description
`bool`	True if the model exists, False if not.

get_config #

get_config(identifier: Union[str, ModelIdentifierField]) -> AnyModelConfig

Get a model's config.

Parameters:

Name	Type	Description	Default
`identifier`	`Union[str, ModelIdentifierField]`	The key or ModelField representing the model.	required

Returns:

Type	Description
`AnyModelConfig`	The model's config.

load #

load(identifier: Union[str, ModelIdentifierField], submodel_type: Optional[SubModelType] = None) -> LoadedModel

Load a model.

Parameters:

Name	Type	Description	Default
`identifier`	`Union[str, ModelIdentifierField]`	The key or ModelField representing the model.	required
`submodel_type`	`Optional[SubModelType]`	The submodel of the model to get.	`None`

Returns:

Type	Description
`LoadedModel`	An object representing the loaded model.

load_by_attrs #

load_by_attrs(name: str, base: BaseModelType, type: ModelType, submodel_type: Optional[SubModelType] = None) -> LoadedModel

Load a model by its attributes.

Parameters:

Name	Type	Description	Default
`name`	`str`	Name of the model.	required
`base`	`BaseModelType`	The models' base type, e.g. `BaseModelType.StableDiffusion1`, `BaseModelType.StableDiffusionXL`, etc.	required
`type`	`ModelType`	Type of the model, e.g. `ModelType.Main`, `ModelType.Vae`, etc.	required
`submodel_type`	`Optional[SubModelType]`	The type of submodel to load, e.g. `SubModelType.UNet`, `SubModelType.TextEncoder`, etc. Only main	`None`

Returns:

Type	Description
`LoadedModel`	An object representing the loaded model.

load_local_model #

load_local_model(model_path: Path, loader: Optional[Callable[[Path], AnyModel]] = None) -> LoadedModelWithoutConfig

Load the model file located at the indicated path

If a loader callable is provided, it will be invoked to load the model. Otherwise, safetensors.torch.load_file() or torch.load() will be called to load the model.

Be aware that the LoadedModelWithoutConfig object has no config attribute

Parameters:

Name	Type	Description	Default
`path`		A model Path	required
`loader`	`Optional[Callable[[Path], AnyModel]]`	A Callable that expects a Path and returns a dict[str\|int, Any]	`None`

Returns:

Type	Description
`LoadedModelWithoutConfig`	A LoadedModelWithoutConfig object.

load_remote_model #

load_remote_model(source: str | AnyHttpUrl, loader: Optional[Callable[[Path], AnyModel]] = None) -> LoadedModelWithoutConfig

Download, cache, and load the model file located at the indicated URL or repo_id.

If the model is already downloaded, it will be loaded from the cache.

If the a loader callable is provided, it will be invoked to load the model. Otherwise, safetensors.torch.load_file() or torch.load() will be called to load the model.

Be aware that the LoadedModelWithoutConfig object has no config attribute

Parameters:

Name	Type	Description	Default
`source`	`str \| AnyHttpUrl`	A URL or huggingface repoid.	required
`loader`	`Optional[Callable[[Path], AnyModel]]`	A Callable that expects a Path and returns a dict[str\|int, Any]	`None`

Returns:

Type	Description
`LoadedModelWithoutConfig`	A LoadedModelWithoutConfig object.

search_by_attrs #

search_by_attrs(name: Optional[str] = None, base: Optional[BaseModelType] = None, type: Optional[ModelType] = None, format: Optional[ModelFormat] = None) -> list[AnyModelConfig]

Search for models by attributes.

Parameters:

Name	Type	Description	Default
`name`	`Optional[str]`	The name to search for (exact match).	`None`
`base`	`Optional[BaseModelType]`	The base to search for, e.g. `BaseModelType.StableDiffusion1`, `BaseModelType.StableDiffusionXL`, etc.	`None`
`type`	`Optional[ModelType]`	Type type of model to search for, e.g. `ModelType.Main`, `ModelType.Vae`, etc.	`None`
`format`	`Optional[ModelFormat]`	The format of model to search for, e.g. `ModelFormat.Checkpoint`, `ModelFormat.Diffusers`, etc.	`None`

Returns:

Type	Description
`list[AnyModelConfig]`	A list of models that match the attributes.

search_by_path #

search_by_path(path: Path) -> list[AnyModelConfig]

Search for models by path.

Parameters:

Name	Type	Description	Default
`path`	`Path`	The path to search for.	required

Returns:

Type	Description
`list[AnyModelConfig]`	A list of models that match the path.

LoggerInterface #

Methods:

Name	Description
`debug`	Logs a debug message.
`error`	Logs an error message.
`info`	Logs an info message.
`warning`	Logs a warning message.

debug #

debug(message: str) -> None

Logs a debug message.

Parameters:

Name	Type	Description	Default
`message`	`str`	The message to log.	required

error #

error(message: str) -> None

Logs an error message.

Parameters:

Name	Type	Description	Default
`message`	`str`	The message to log.	required

info #

info(message: str) -> None

Logs an info message.

Parameters:

Name	Type	Description	Default
`message`	`str`	The message to log.	required

warning #

warning(message: str) -> None

Logs a warning message.

Parameters:

Name	Type	Description	Default
`message`	`str`	The message to log.	required

ConfigInterface #

Methods:

Name	Description
`get`	Gets the app's config.

get #

get() -> InvokeAIAppConfig

Gets the app's config.

Returns:

Type	Description
`InvokeAIAppConfig`	The app's config.

UtilInterface #

Methods:

Name	Description
`flux_step_callback`	The step callback emits a progress event with the current step, the total number of
`is_canceled`	Checks if the current session has been canceled.
`sd_step_callback`	The step callback emits a progress event with the current step, the total number of
`signal_progress`	Signals the progress of some long-running invocation. The progress is displayed in the UI.

flux_step_callback #

flux_step_callback(intermediate_state: PipelineIntermediateState) -> None

The step callback emits a progress event with the current step, the total number of steps, a preview image, and some other internal metadata.

This should be called after each denoising step.

Parameters:

Name	Type	Description	Default
`intermediate_state`	`PipelineIntermediateState`	The intermediate state of the diffusion pipeline.	required

is_canceled #

is_canceled() -> bool

Checks if the current session has been canceled.

Returns:

Type	Description
`bool`	True if the current session has been canceled, False if not.

sd_step_callback #

sd_step_callback(intermediate_state: PipelineIntermediateState, base_model: BaseModelType) -> None

The step callback emits a progress event with the current step, the total number of steps, a preview image, and some other internal metadata.

This should be called after each denoising step.

Parameters:

Name	Type	Description	Default
`intermediate_state`	`PipelineIntermediateState`	The intermediate state of the diffusion pipeline.	required
`base_model`	`BaseModelType`	The base model for the current denoising step.	required

signal_progress #

signal_progress(message: str, percentage: float | None = None, image: Image | None = None, image_size: tuple[int, int] | None = None) -> None

Signals the progress of some long-running invocation. The progress is displayed in the UI.

If a percentage is provided, the UI will display a progress bar and automatically append the percentage to the message. You should not include the percentage in the message.

Example

total_steps = 10
for i in range(total_steps):
    percentage = i / (total_steps - 1)
    context.util.signal_progress("Doing something cool", percentage)

If an image is provided, the UI will display it. If your image should be displayed at a different size, provide a tuple of (width, height) for the image_size parameter. The image will be displayed at the specified size in the UI.

For example, SD denoising progress images are ⅛ the size of the original image, so you'd do this to ensure the image is displayed at the correct size:

# Calculate the output size of the image (8x the progress image's size)
width = progress_image.width * 8
height = progress_image.height * 8
# Signal the progress with the image and output size
signal_progress("Denoising", percentage, progress_image, (width, height))

If your progress image is very large, consider downscaling it to reduce the payload size and provide the original size to the image_size parameter. The PIL thumbnail method is useful for this, as it maintains the aspect ratio of the image:

# `thumbnail` modifies the image in-place, so we need to first make a copy
thumbnail_image = progress_image.copy()
# Resize the image to a maximum of 256x256 pixels, maintaining the aspect ratio
thumbnail_image.thumbnail((256, 256))
# Signal the progress with the thumbnail, passing the original size
signal_progress("Denoising", percentage, thumbnail, progress_image.size)

Parameters:

Name	Type	Description	Default
`message`	`str`	A message describing the current status. Do not include the percentage in this message.	required
`percentage`	`float \| None`	The current percentage completion for the process. Omit for indeterminate progress.	`None`
`image`	`Image \| None`	An optional image to display.	`None`
`image_size`	`tuple[int, int] \| None`	The optional size of the image to display. If omitted, the image will be displayed at its original size.	`None`

BoardsInterface #

Methods:

Name	Description
`add_image_to_board`	Adds an image to a board.
`create`	Creates a board.
`get_all`	Gets all boards.
`get_all_image_names_for_board`	Gets all image names for a board.
`get_dto`	Gets a board DTO.

add_image_to_board #

add_image_to_board(board_id: str, image_name: str) -> None

Adds an image to a board.

Parameters:

Name	Type	Description	Default
`board_id`	`str`	The ID of the board to add the image to.	required
`image_name`	`str`	The name of the image to add to the board.	required

create #

create(board_name: str) -> BoardDTO

Creates a board.

Parameters:

Name	Type	Description	Default
`board_name`	`str`	The name of the board to create.	required

Returns:

Type	Description
`BoardDTO`	The created board DTO.

get_all #

get_all() -> list[BoardDTO]

Gets all boards.

Returns:

Type	Description
`list[BoardDTO]`	A list of all boards.

get_all_image_names_for_board #

get_all_image_names_for_board(board_id: str) -> list[str]

Gets all image names for a board.

Parameters:

Name	Type	Description	Default
`board_id`	`str`	The ID of the board to get the image names for.	required

Returns:

Type	Description
`list[str]`	A list of all image names for the board.

get_dto #

get_dto(board_id: str) -> BoardDTO

Gets a board DTO.

Parameters:

Name	Type	Description	Default
`board_id`	`str`	The ID of the board to get.	required

Returns:

Type	Description
`BoardDTO`	The board DTO.

Invocation API#

Mixins#

WithMetadata#

WithBoard#

InvocationContext #

ImagesInterface #

get_dto #

get_metadata #

get_path #

get_pil #

save #

TensorsInterface #

load #

save #

ConditioningInterface #

load #

save #

ModelsInterface #

download_and_cache_model #

exists #

get_config #

load #

load_by_attrs #

load_local_model #

load_remote_model #

search_by_attrs #

search_by_path #

LoggerInterface #

debug #

error #

info #

warning #

ConfigInterface #

get #

UtilInterface #

flux_step_callback #

is_canceled #

sd_step_callback #

signal_progress #

BoardsInterface #

add_image_to_board #

create #

get_all #

get_all_image_names_for_board #

get_dto #

`WithMetadata`#

`WithBoard`#