Migrating from Ollama to Text Generation WebUI: What Changed

I spent most of today moving my local LLM setup from Ollama to Text Generation WebUI. Not because Ollama broke or stopped working—it didn’t—but because I kept bumping into the same friction points. I wanted to see if Text Generation WebUI actually solved them or if I was just chasing a different set of problems.

Spoiler: it mostly did. But there’s a cost to that flexibility.

Why I Left Ollama

Ollama was fine. It was simple, it pulled models from a registry, it ran them, it had an API. I could curl it from scripts. The model management felt good—ollama pull mistral and you’re done. No thinking about quantization levels, no hunting GitHub releases for the right file format.

The problem wasn’t Ollama’s fault so much as the shape of what I was trying to do.

First: model format rigidity. Ollama expects GGUF. Full stop. I had a couple of GPTQ models lying around that I’d already quantized myself, and there was no clean way to load them into Ollama without re-quantizing as GGUF. That’s not a huge deal in theory, but in practice it meant disk space and time I didn’t want to spend.

Second, and this is the one that actually got me moving: I wanted better control over inference parameters at load time without editing config files. Ollama has some environment variable controls, but if you wanted to swap between different context lengths, sampling strategies, or system prompts without restarting, you were basically out of luck. You could hot-swap the model through the API, but the generation parameters felt locked in.

Third—and I’ll admit this is a “nice to have” complaint—I wanted to see what the model was actually doing during inference. Token counts, probabilities, beam search decisions. Ollama gives you nothing on that front. It’s a black box by design. For a homelab setup where I’m the only user, that opacity bothered me more than it probably should.

So I spent an evening looking at alternatives. Text Generation WebUI kept showing up in every comparison thread, described as the “Swiss Army knife” option. I was skeptical. “Swiss Army knife” usually means “does everything badly.” But I decided to try it.

The Install and First Run

I cloned the repo and ran the one-liner install script on my Ubuntu box. Took about twelve minutes. Python venv, a bunch of pip installs, and it dropped a launcher script in the home directory. That was faster than I expected, honestly.

The first UI load was… a lot. The Gradio interface opened up with what felt like fifty different tabs and options. Chat, Instruct, Notebook modes. Parameters I’d never heard of. A sidebar full of settings. No README in the UI itself that says “start here.” I stared at it for a moment wondering if I’d made a mistake.

I didn’t. It was just me not knowing what I was looking at yet.

To load a model, you point it at a directory, select your backend, and hit “Load.” I pointed it at my GPTQ model (Mistral 7B, 5-bit, about 4.5GB), and it loaded in seconds. The UI immediately showed me context length options, sampling parameters, a quantization method dropdown, and a bunch of other controls I could actually interact with in real time.

That’s when I got it. Ollama abstracts away the entire machinery. Text Generation WebUI puts all the machinery on the dashboard.

What I Actually Use vs. What’s There

The Chat tab is where I spend most of my time. You get a conversation interface with character/system prompt support, message history management, and token counting right in the interface. Nothing revolutionary, but it’s clean and functional.

The real difference is in the settings. Every sampling parameter you can think about is right there: temperature, top-p, top-k, repetition penalty, min-p, typical penalty. Each one has a slider and a text input. I can adjust them without restarting the model or editing YAML. That alone justifies the migration for me.

I haven’t touched the LoRA support yet, or the extensions, or the notebook mode. Those exist, they’re apparently robust, but they’re not why I’m here. I’m here because I wanted to stop fighting with Ollama’s constraints, and Text Generation WebUI doesn’t have the same constraints.

The API is there too. Running on port 5000 by default, compatible with OpenAI’s format so I can point other tools at it. I haven’t switched my Home Assistant or Node-RED integrations over yet, but the option is there without any special configuration.

What I Miss About Ollama

Ollama’s model library was genuinely nice. You didn’t have to think. ollama pull neural-chat and you got something reasonable. Model management was a solved problem. With Text Generation WebUI, you’re hunting HuggingFace or TheBloke’s repo, checking compatibility matrices, making sure you’re downloading the right quantization for your hardware.

It’s a five-minute task, but it’s five minutes of thinking that Ollama didn’t require.

Also—and this surprised me—Ollama’s performance on CPU-only inference was noticeably better. I benchmarked both on the same hardware with the same model, and Ollama’s optimizations won. Text Generation WebUI gives you more control, but that control comes with overhead. For my GPU-equipped setup it doesn’t matter. If I were running on CPU, I might feel differently.

The cleanup story is also cleaner in Ollama. Pulling a new model just pulls it, it’s tracked, `ollama rm model-name` removes it. In Text Generation WebUI you’re managing folders, remembering where models live on disk, handling deletion yourself. Not hard, but less elegant.

The Speed and Size Questions

Text Generation WebUI is heavier on disk. The install itself is about 1.2GB with Python deps. Ollama was maybe 300MB. That’s not a real problem in a homelab, but it’s true.

Speed is more interesting. Token generation per second is basically identical—I’m hitting the same VRAM bottleneck either way. Load time for the interface is slightly slower in Text Generation WebUI because there’s more UI to render. The actual model inference? No practical difference on my RTX 3070.

Startup time is roughly the same for both. The launcher script takes a moment, the Gradio UI takes a moment to initialize, and then you’re ready.

The Real Question: Was It Worth It?

Honestly, it depends on what you’re doing. If you just want to run models with an API and don’t care about tweaking parameters on the fly, Ollama is less friction. Stick with it. It’s good software.

If you want to experiment—different quantization methods, parameter tuning, loading multiple model formats, maybe plugging in LoRAs—Text Generation WebUI pays for itself quickly. The flexibility is real. You’re not fighting the tool, you’re using the tool. That’s worth an afternoon of setup.

What I wasn’t expecting was how much clearer the entire process becomes when you can see all the knobs. I understand my inference pipeline better now, even if I don’t touch most of the settings. That understanding has value beyond just the technical specifications.

I’m not worried I made the wrong choice. But I’m also aware that in another six months I might build some automation that makes Ollama’s simplicity appealing again. Homelab stuff is like that. You pick the right tool for today, and today’s right answer is Text Generation WebUI.

FAQ

Can I run Text Generation WebUI without a GPU?

Yes, but it’ll be slow. CPU inference works through llama.cpp backend, and you’ll get acceptable speeds for small models (7B range) if you’re patient. With a GPU it’s fast enough to be interactive.

How much RAM does Text Generation WebUI need?

The UI itself uses about 2-3GB. Add another 4-8GB depending on your model size and context length. A 7B model in 4-bit quantization takes roughly 4GB VRAM plus headroom.

Does Text Generation WebUI work with OpenAI-compatible APIs?

Yes. It includes an OpenAI-compatible API endpoint on port 5000 by default. You can point tools like Home Assistant, LangChain, or anything else that speaks OpenAI API at it directly.

What model formats does Text Generation WebUI support?

GGUF, GPTQ, AWQ, EXL2, HQQ, and raw PyTorch formats. It’s the broadest format support in the space, which is why format compatibility isn’t a practical concern.

Is Text Generation WebUI better than Ollama?

They solve different problems. Ollama prioritizes simplicity; Text Generation WebUI prioritizes control. If you want to set a model and forget it, Ollama wins. If you want to tune and experiment, Text Generation WebUI wins.

Explore Text Generation WebUI in our AI Homelab Toolkit.