Skip to main content

SQS triggers

Windmill can connect to an SQS queue and trigger runnables (scripts, flows) in response to messages received.
SQS triggers is a self-hosted Enterprise feature.

For more details on SQS, see the AWS SQS Documentation.

SQS triggers

How to use​

  • Pick an AWS resource

    • Select an existing AWS resource or create a new one.
    • The AWS resource must have permissions to interact with SQS.

  • Select the runnable to execute

    • Choose the runnable (script or flow) that should be executed when a message arrives in the queue.
    • The message will be passed as a JSON object to the runnable.

  • Provide an SQS queue URL

    • Enter the Queue URL of the SQS queue that should trigger the runnable.
    • You can find the Queue URL in the AWS Management Console under SQS.
    • For more details, see the SQS Queue URL Documentation.

  • Choose (optional) message attributes

    • Specify which message attributes should be included in the triggered event.
    • These attributes can carry metadata, such as sender information or priority levels.
    • For more details, see the SQS Message Attributes Documentation.

Example​

Below are code examples demonstrating how to handle SQS messages in your Windmill scripts. You can either process messages directly in a basic script or use a preprocessor for more advanced message handling and transformation before execution.

Basic script example​

export async function main(msg: string) {
  // do something with the message
}

Using a preprocessor​

If you use a preprocessor, the preprocessor function receives an SQS message with the following fields:

Field descriptions​

  • queue_url: The URL of the SQS queue that received the message.
  • message_id: A unique identifier assigned to each message by SQS.
  • receipt_handle: A token used to delete the message after processing.
  • attributes: Metadata attributes set by SQS, such as SentTimestamp.
  • message_attributes: User-defined attributes that can be attached to the message.
    • string_value: The string representation of the attribute value.
    • data_type: The data type of the attribute (e.g., "String", "Number", "Binary").
    • More details on message attributes
export async function preprocessor(
    msg: string,
    wm_trigger: {
        kind: "sqs",
        sqs: {
            queue_url: string,
            message_id?: string,
            receipt_handle?: string,
            attributes: Record<string, string>,
            message_attributes?: Record<string, {
                string_value?: string,
                data_type: string
            }>
        }
    },
) {
    // assuming the message is a JSON object
    const data = JSON.parse(msg);
    
    return {
        content: data.content,
        metadata: {
            sentAt: wm_trigger.sqs.attributes.SentTimestamp,
            messageId: wm_trigger.sqs.message_id
        }
    };
}

export async function main(content: string, metadata: { sentAt: string, messageId: string }) {
    // Process transformed message data
    console.log(`Processing message ${metadata.messageId} sent at ${metadata.sentAt}`);
    console.log("Content:", content);
}