ComfyUI + Home Assistant: Triggering image generation from automations

I needed my homelab to generate images when specific conditions fired in Home Assistant. Not a gallery refresh or a scheduled batch job—actual triggering from automations. ComfyUI’s node system is overkill for simple tasks, but it’s exactly what you want when you need repeatability and control. Home Assistant already runs 24/7 on my network, so the integration felt natural. Turned out to be more friction than I expected.

Why this matters

ComfyUI runs workflows. Home Assistant runs automations. Neither tool is built to talk to the other, but both expose HTTP APIs. The gap between them is closure. Trigger a workflow from a sensor state change. Pass image paths back into Home Assistant for notifications or dashboard display. Once wired, it becomes another utility you stop thinking about.

The alternative is running ComfyUI standalone and manually uploading prompts. I did that for three weeks before the tedium broke me.

Setting up the webhook receiver in ComfyUI

ComfyUI doesn’t have native webhook support. It has an API that accepts POST requests with JSON payloads describing workflows. The trick is converting Home Assistant automation triggers into those payloads.

Start with a minimal ComfyUI workflow. Something that takes a prompt text input, sends it to Stable Diffusion, and saves the output. Export that workflow as JSON through the UI. File menu, Export. This becomes your template.

The template will have a structure like this:

{
  "1": {
    "inputs": {
      "text": "a cat sitting on a table"
    },
    "class_type": "CLIPTextEncode (Positive)"
  },
  "2": {
    "inputs": {
      "text": ""
    },
    "class_type": "CLIPTextEncode (Negative)"
  },
  "3": {
    "inputs": {
      "seed": 12345,
      "steps": 20,
      "cfg": 8.0,
      "sampler_name": "euler",
      "scheduler": "normal",
      "denoise": 1.0,
      "model": ["4", 0],
      "positive": ["1", 0],
      "negative": ["2", 0],
      "latent_image": ["5", 0]
    },
    "class_type": "KSampler"
  }
}

The node IDs (“1”, “2”, “3”) are what matter. Home Assistant will POST this structure to ComfyUI’s API endpoint and the prompt text field becomes your injection point.

Expose the ComfyUI API on your network. By default it listens on 127.0.0.1:8188. If Home Assistant runs on a different machine, you need to bind to 0.0.0.0. Add this to your launch command or your docker-compose:

version: '3.8'
services:
  comfyui:
    image: ghcr.io/comfyanonymous/comfyui:latest
    ports:
      - "8188:8188"
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
    volumes:
      - ./models:/root/.cache/huggingface
      - ./output:/root/ComfyUI/output
    command: python main.py --listen 0.0.0.0

That --listen 0.0.0.0 flag is the key. Now the API accepts requests from other machines on your network.

Calling ComfyUI from Home Assistant

Home Assistant has a REST command integration. This is what sends the trigger payload to ComfyUI.

In your configuration.yaml:

rest_command:
  generate_image:
    url: 'http://comfyui.local:8188/prompt'
    method: POST
    payload: '{"client_id": "homeassistant", "prompt": {{ prompt_payload }} }'
    content_type: 'application/json'

That’s not quite right. You’re not sending the full workflow structure. Let me correct that. The payload needs to be your entire workflow JSON with the text field swapped in:

rest_command:
  generate_image:
    url: 'http://comfyui.local:8188/prompt'
    method: POST
    payload: '{{ workflow_json }}'
    content_type: 'application/json'

The workflow_json is passed as a template variable from your automation. Build the JSON string in a template first:

automation:
  - alias: Generate image from motion
    trigger:
      platform: state
      entity_id: binary_sensor.front_porch_motion
      to: 'on'
    action:
      - service: rest_command.generate_image
        data_template:
          workflow_json: >-
            {"1": {"inputs": {"text": "security camera view of a front porch at night"}, "class_type": "CLIPTextEncode (Positive)"}, "2": {"inputs": {"text": ""}, "class_type": "CLIPTextEncode (Negative)"}, "3": {"inputs": {"seed": {{ range(0, 999999999) | random }}, "steps": 20, "cfg": 8.0, "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, "model": ["4", 0], "positive": ["1", 0], "negative": ["2", 0], "latent_image": ["5", 0]}, "class_type": "KSampler"}}

That’s unwieldy. I eventually built a custom REST endpoint in a separate Python service that accepts simple parameters and constructs the full workflow. Saved me from wrestling with Jinja2 template escaping.

The bit that tripped me up

ComfyUI queues jobs. When you POST to /prompt, it returns a prompt_id. The image doesn’t exist yet. It takes 10-30 seconds depending on your hardware. Home Assistant fires the command and moves on. You don’t get immediate feedback.

I tried polling the output directory from Home Assistant. Worked for five days then started missing images because the polling interval was too loose. I switched to websockets so ComfyUI could notify Home Assistant when an image was ready. That required a small Python bridge I wrote and now maintain.

Simpler approach: save the output to a predictable path and have Home Assistant check for it on a 5-second loop with a timeout of 120 seconds. Not elegant but it works and doesn’t require extra infrastructure.

- service: homeassistant.update_entity
  entity_id: sensor.comfyui_last_image
  data_template:
    timestamp: "{{ now().isoformat() }}"

Persistence and security

Keep the ComfyUI API private. It runs unauthenticated by default. Put it behind a reverse proxy with auth if it’s accessible from outside your network. I use Traefik with basic auth and a self-signed cert.

Store your workflow templates as JSON files in version control. They’re cheap to store and they’re your documentation. When you tweak a workflow node, version it. I learned this the hard way after losing a working config during a ComfyUI update.

Mount the output directory as a persistent volume. ComfyUI will fill it with generated images. Home Assistant can read them for dashboards or use them in notifications. The volume survives container restarts, which matters.

This setup has run for four months now. Most of the friction came from not understanding ComfyUI’s async queue model. The integration itself is straightforward once you accept that you’re polling for results instead of getting callbacks. It works, and it stops working only when I break something in my automation YAML.

Explore ComfyUI in our AI Homelab Toolkit.