How Flowise works under the hood: the architecture explained

When you drag a node onto a Flowise canvas and connect it to another, you’re not just drawing a picture. Behind that visual interface sits an execution engine that’s translating your flow into a directed acyclic graph (DAG), managing state across distributed node execution, and coordinating API calls to whatever LLM backend you’ve pointed it at. Understanding how Flowise works under the hood matters more than you’d think, especially when you’re self-hosting and something breaks at 2 AM.

The Core Architecture: Nodes, Flows, and the Graph Engine

Flowise is built on a node-based architecture. Every component you see on the canvas—an LLM node, a vector store retrieval node, a prompt template, a tool wrapper—is fundamentally a discrete unit with defined inputs, outputs, and internal logic. When you connect these nodes together, you’re creating a dependency graph that describes data flow.

The actual execution happens in the backend, which is a Node.js application running Express. The frontend is React, and it’s responsible only for rendering the canvas and letting you wire things together. When you click “save,” the canvas state gets serialized to JSON and stored in a database (by default, SQLite, but it can be Postgres or MySQL). That JSON contains the node definitions, their parameters, and the connection metadata between them.

Here’s the thing that actually matters: the JSON isn’t just for storage. When you trigger a flow—via API, chat interface, or webhook—Flowise reads that JSON and rebuilds the execution DAG from scratch. It doesn’t keep the graph in memory. Every invocation parses the flow definition, instantiates the nodes in dependency order, and runs them. This sounds inefficient, but it’s actually the right call for a self-hosted tool where flows might change mid-session and you don’t have the infrastructure overhead of a massive SaaS platform.

The execution engine itself lives in a TypeScript layer that wraps the popular LangChain library. Flowise doesn’t reinvent the wheel for LLM orchestration—it uses LangChain’s Runnable interface as the backbone. Each node type (LLM, prompt, retriever, tool) gets wrapped into a LangChain Runnable, and then those are chained together using LangChain’s composition operators. The visual abstraction is just a different UI on top of something you could build directly in Python or JavaScript.

Data Flow: From Canvas Click to LLM Response

Let’s trace what happens when you have a simple three-node flow: a chat input node, connected to an LLM node, connected to an output node. You type a message. The frontend posts to the Flowise API endpoint for that flow. The backend receives the request with the user’s input message.

It loads the flow definition from the database. It instantiates each node in topological order—dependencies first. The chat input node doesn’t actually do anything except mark the incoming data as “message.” The LLM node reads its inputs (which include the message and a prompt template you configured), formats them, and calls out to your configured LLM—OpenAI’s API, a local Ollama instance, Hugging Face, whatever you’ve set up. The response comes back. The output node (which is basically a pass-through) returns it to the user.

This is where self-hosting becomes real. If you’ve pointed Flowise at Ollama running on the same machine, the request goes over localhost:11434. If it’s OpenAI, it goes out to their API. If you’ve configured a vector store (like Pinecone or a self-hosted Weaviate), the retrieval step queues a search and waits for the results. All of this is happening in the same Node.js event loop, so concurrency matters. By default, Flowise spins up a single worker process, but in production, you’d typically run it behind multiple replicas or a queue system.

One thing that surprised me when I first ran Flowise: memory usage for longer conversations. If you’re keeping conversation history in the flow (which most chatbot flows do), the token count of that history grows with every turn. Flowise loads the entire history into the prompt context each time—no windowing, no smart summarization by default. You have to manually add a node for that, or watch your token spend climb fast if you’re using a paid API. It’s not a flaw exactly, but it’s a gotcha.

Node Types and Their Internal Implementations

Flowise ships with about thirty node types out of the box, but they fall into a few categories. Understanding what each category does helps you reason about where the load actually sits.

Input/Output nodes are lightweight wrappers. They don’t process anything; they just mark data boundaries for the flow. The chat input node accepts text, the file input node watches a directory or accepts multipart uploads.

LLM nodes are where most of the work happens. These are wrappers around actual LLM APIs. The OpenAI node, for instance, initializes the OpenAI SDK with your API key, manages temperature and max_tokens parameters, and handles retries and error rates. A local Ollama node does the same thing but points to your local instance. These nodes are stateless—they don’t maintain context between calls.

Memory and retrieval nodes are more interesting. The vector store retrieval node takes your query, embeds it (using an embedding model you’ve configured, like OpenAI’s text-embedding-3-small or a local sentence-transformer), and searches a vector database. That database could be Pinecone, Weaviate, Milvus, or even a simple in-memory vector store for testing. The embedding step is often the bottleneck in RAG pipelines. Embedding a 500-token document locally on a homelab CPU takes time. Embedding remotely means an extra API call.

Tool nodes wrap external functions or APIs. A tool node for an HTTP request, for instance, takes URL and payload templates, substitutes values from flow context, and makes the actual request. Tool nodes are how Flowise implements agent loops—an LLM node can be configured to use tools, and if the LLM outputs a tool call (in the right format), the tool node executes it and loops the response back to the LLM.

All of these node types compile down to LangChain Runnables. That’s the abstraction layer that lets Flowise stay language-agnostic and swap backends without rewriting the core engine.

State Management and Context Passing

Context is how data moves through the graph. When a node executes, it receives input from its dependencies and produces output. That output becomes available as input to any node that depends on it.

Flowise implements this with a context object that flows through the entire execution. At the start of a run, context contains the user’s input message plus any parameters you’ve manually set on nodes (like system prompts or API keys). As each node runs, it reads from context, does its work, and updates context with its output. By the time execution reaches the final node, context has accumulated all the intermediate results.

This is transparent if your flow is simple. But if you’ve got ten nodes and you want to debug why node seven is getting the wrong input, you’ll need to inspect the context directly. Flowise has basic logging for this, but it’s not deep. You end up adding debug output nodes (which just print to stdout) strategically in your flow.

There’s also a session concept for maintaining conversation history. When you configure a chat flow with a memory node, Flowise stores message history in the database (keyed by session ID) and loads the relevant history for each new turn. The memory node itself is a wrapper around LangChain’s memory implementations—BufferMemory for simple recall, or SummaryMemory if you want it to periodically summarize old turns to conserve tokens.

The API Layer and Trigger Mechanisms

Once you’ve built and saved a flow, Flowise exposes it via REST API. Every flow gets a unique endpoint. You can POST to that endpoint with input data (usually a message and optional metadata), and it returns the flow output.

By default, Flowise runs on port 3000, and your flow might be at /api/v1/prediction/abc123def456. The request body looks like:

{
  "question": "What's the capital of France?",
  "chatHistory": [
    {"role": "user", "content": "Hi"},
    {"role": "assistant", "content": "Hello!"}
  ],
  "sessionId": "user-123"
}

The response is the flow’s final output—usually a text response from the LLM, but it could be structured JSON if you’ve configured that.

You can also invoke flows via webhooks (incoming), schedule them with cron-like triggers (if you’re running the paid cloud version), or chain them together. The webhook trigger is handy for homelab setups: your Home Assistant automation posts to a Flowise webhook, which triggers a flow, which might call out to a tool, and the result comes back to Home Assistant.

There’s no built-in queue system by default. If twenty requests hit your flow at once, they all execute in parallel (limited by Node.js event loop concurrency and whatever resource limits your host has). If you’re self-hosting and expecting traffic, you’d front Flowise with a reverse proxy like Nginx and possibly a job queue like Bull to handle bursting.

Storage and Persistence

Flowise needs somewhere to store flow definitions, session history, chat messages, and uploaded files. The database layer is pluggable. By default, you get SQLite, which is fine for a single-user or small-team homelab. It’s a file at /root/.flowise/database.db (inside the Docker container) or wherever you mount your volume.

For multi-user setups or higher reliability, you’d configure a Postgres connection string. Flowise uses TypeORM as its ORM layer, so migrations and schema management happen automatically on startup. You don’t have to manually create tables.

Uploaded files (documents for RAG, for instance) are stored in a files directory. If you’re running Flowise in Docker, you need to mount a volume there or they’ll disappear when the container stops. This is a common gotcha. I lost an entire set of ingested PDFs once because I didn’t think through the volume mounts. Now I always do:

docker run -v ~/.flowise:/root/.flowise n  -v ~/.flowise/files:/root/.flowise/files n  -p 3000:3000 n  flowiseai/flowise

Vector embeddings, if you’re using an external vector store, don’t live in Flowise itself—they live in your vector database. But if you’re using the in-memory vector store for testing, that’s ephemeral and disappears on restart.

Performance Bottlenecks and Where They Actually Are

If you’ve built a flow and you’re waiting too long for responses, the bottleneck is almost never Flowise itself. It’s one of three things: your LLM (if it’s slow to respond or you’re on a limited API rate), your vector store (if retrieval is doing a large search), or your network (if you’re making external API calls).

Flowise is pretty efficient with what it does. A simple flow with no external calls executes in microseconds. The overhead of parsing the JSON, instantiating the nodes, and wiring them together is minimal compared to, say, calling OpenAI’s API.

One real bottleneck is embedding for RAG. If you’re ingesting documents with Flowise’s document loader nodes, you’re embedding them synchronously by default. A thousand documents, each embedded with OpenAI’s API, will take an hour or more and cost money. If you’re using a local embedding model through Ollama, it’s slower but free—maybe a hundred documents an hour on a decent CPU. I usually ingest documents offline and push them to the vector store directly to avoid tying up the Flowise instance.

Another subtle one: if you’ve got a loop in your flow where an agent calls a tool, gets a response, and loops back to the LLM, each iteration calls your LLM and potentially its embedding model. If your LLM is slow, you’ll feel every iteration. With a fast local model, it’s fine. With OpenAI, you’re paying per call.

Security and Isolation Considerations

Flowise stores API keys in the database in plain text (or at least, that’s the simplest configuration). If you’re running this on a homelab machine with other people’s access, you should know that. In production environments, Flowise supports environment variables for sensitive values, so you can set OPENAI_API_KEY once at startup and reference it in nodes rather than storing the actual key in a flow.

The API endpoint for your flow is public by default. Anyone who knows the URL can call it. If that matters (and it should), put it behind authentication. The simplest approach is a reverse proxy with basic auth or API keys.

Tool nodes that make HTTP calls can be a security concern if users can edit flows. A user could add a tool node that makes arbitrary requests to internal services. If Flowise is on your internal network and can reach other machines, that’s a lateral movement risk. For a personal homelab, less of an issue. For a multi-user setup, you’d want to restrict what tool nodes can do.

The Docker Reality

Most people run Flowise in Docker, and for good reason. The Node.js dependencies are substantial and fragile across platforms. Docker isolates that mess. The official image pulls the latest code on each release, which is convenient but means you’re opinionated about how Flowise runs—it uses port 3000, expects volumes at specific paths, and runs as root inside the container (which is fine for homelab but not ideal for production).

A typical Docker Compose setup looks like this:

version: '3'
services:
  flowise:
    image: flowiseai/flowise:latest
    environment:
      - PORT=3000
      - DATABASE_PATH=/root/.flowise
      - LOG_LEVEL=debug
    ports:
      - "3000:3000"
    volumes:
      - ~/.flowise:/root/.flowise
    networks:
      - flowise-net

networks:
  flowise-net:
    driver: bridge

From there, you’d typically add Postgres if you want to, connect it to Ollama or your LLM, and point any external tools at the right services. The networking is straightforward as long as you remember that services on different networks can’t reach each other.

I’ve had Flowise containers get stuck in a broken state where they wouldn’t start—usually a corrupted database file or a bad environment variable. The Docker image doesn’t have great error reporting in those cases. You end up shelling into the container and checking logs manually.

Building anything non-trivial with Flowise involves time spent on operational details that have nothing to do with the actual AI logic. That’s just the cost of self-hosting, but it’s worth naming: Flowise itself is elegant, but the infrastructure around it can be finicky.

What Flowise Actually Is (And Isn’t)

After diving into this, here’s my honest take: Flowise is a visual interface for LLM workflows that abstracts away most of the boilerplate. It’s not a magic box that generates applications. It’s not simpler than writing code if you’re already comfortable with code. What it does is let you iterate on AI workflows faster than you would with a code-based framework, and it lets non-engineers build and tweak flows without touching terminal or writing logic.

The architecture is sound. The execution engine is solid. The real wins are in iteration speed and the visual feedback loop. The real friction points are in operationalization—hooking it into your broader system, managing state across flows, scaling it past toy projects.

If you’re thinking about using Flowise, you’re not just adopting a tool. You’re adopting a workflow and a set of architectural patterns that lean heavily on external services (LLMs, vector stores, APIs). Understanding how those pieces fit together—which is what the internals tell you—matters more than understanding Flowise’s code. Flowise is the coordinator. Everything else is the actual work.

FAQ

How does Flowise execute flows if you can’t see the code?

Flowise compiles your visual flow into a JSON graph, then converts that into chained LangChain Runnables at execution time. Every time you run the flow, it rebuilds the graph from the JSON and executes it node-by-node in dependency order. You’re not looking at the code, but the execution logic is plain once you understand it’s wrapping LangChain.

Can I run Flowise without an internet connection?

Yes, as long as your LLM is also local (like Ollama) and your vector store doesn’t need external calls. The Flowise application itself doesn’t phone home. However, if you’re using OpenAI, Hugging Face, or any cloud API, you’ll need internet for those calls.

What happens to my API keys in Flowise?

By default, they’re stored in plain text in the Flowise database. For security, pass them as environment variables at startup instead—Flowise will reference the environment variable in your flow nodes rather than storing the actual key.

Does Flowise support running multiple flows in parallel?

Yes. The Node.js event loop handles concurrent requests, so if you send ten API calls to your Flowise instance at once, all ten will execute in parallel (constrained by available CPU and memory). For production traffic, you’d run multiple Flowise instances behind a load balancer.

How much does it cost to run Flowise on a homelab?

Flowise itself is free and open source. Your costs come from external services: OpenAI API calls, Pinecone vector store fees, etc. If you use only local models (Ollama) and open-source vector stores (Milvus, Weaviate), the only cost is your electricity.

Explore Flowise in our AI Homelab Toolkit.