The cloud is optional. Four open-source projects run a language model on hardware you already own, with no per-token bill: Oobabooga's TextGen, KoboldCpp, llama.cpp and vLLM. They overlap, but they are not interchangeable. What follows is what each project documents about installing it, feeding it a model, fitting it in memory, and paying for it — and nothing the documentation does not say.

The four backends, and who each is for

  • Oobabooga TextGen (formerly text-generation-webui) — a desktop app for local LLMs with text, vision, tool-calling and OpenAI/Anthropic-compatible APIs. The project calls it "100% offline and private, with zero telemetry, external resources, or remote update requests."
  • KoboldCpp — a single executable built on llama.cpp for running GGUF models on CPU or GPU, with the KoboldAI Lite web UI, world info, characters and Tavern card import bundled.
  • llama.cpp — the C/C++ engine underneath both of the above: MIT-licensed, no dependencies, 1.5-bit to 8-bit quantisation, CPU+GPU hybrid inference.
  • vLLM — a high-throughput serving library, not a desktop app. PagedAttention, continuous batching, chunked prefill, prefix caching, and an OpenAI-compatible server plus Anthropic Messages and gRPC.

On 9 November 2026 the repositories showed 47.7k stars for Oobabooga (AGPL-3.0, 6.0k forks), 12.2k for KoboldCpp (AGPL-3.0) and 133.8k for llama.cpp (MIT). vLLM's docs describe a project that originated at UC Berkeley's Sky Computing Lab, lists 2,000+ contributors and supports 200+ Hugging Face model architectures.

Hardware: the knobs the projects document

None of these projects publishes a first-party table mapping quantisation to VRAM. What they document is the set of controls that decides whether a model fits. llama.cpp supports integer quantisation from 1.5-bit to 8-bit, and KoboldCpp's recommended starter models are GGUF quants (Q4_K_S/Q4_K_M) — a fair pointer to the range people actually run.

In KoboldCpp, --gpulayers N sets GPU offload: the default -1 is autofit, 0 is CPU-only. When VRAM runs out the README says to "try fewer GPU layers, a smaller model, or a shorter context." Context is set with --contextsize N, and "larger contexts need more memory." A GPU is optional: KoboldCpp has a -nocuda build, Oobabooga has a CPU-only build and llama.cpp runs on CPU. As KoboldCpp puts it, "A dedicated GPU is optional; the model size and context length determine how much memory you need." Speed on CPU depends on hardware and model, and the projects promise no number.

For a rough estimate, Oobabooga links a GGUF Memory Calculator on Hugging Face (accurate-gguf-vram-calculator). It is the closest thing to an official sizing tool across the four projects.

Installing each one

Oobabooga's desktop route is download, unzip, double-click textgen. Portable builds cover Linux, Windows and macOS with CUDA, Vulkan, ROCm and CPU-only variants, dependencies included, and GGUF models work. ExLlamaV3, Transformers, training and image generation need the full install instead — about 10GB of disk and a PyTorch download. Drop a GGUF into user_data/models and the UI finds it; for auto-load, put --model my-model.gguf in user_data/CMD_FLAGS.txt.

KoboldCpp is one executable, no installer. Builds are available for Windows (koboldcpp.exe, plus -nocuda and -oldpc), Linux x64 and macOS ARM64; Intel Macs need a source build, and Android works through Termux. Quick start: download the exe, download a GGUF, select it in the GGUF Text Model field, launch, open http://localhost:5001.

llama.cpp installs with the project's script (curl -LsSf https://llama.app/install.sh | sh, or the PowerShell installer), Docker, pre-built binaries or a source build. llama cli -hf <model-GGUF> runs a model from Hugging Face; llama serve -hf <model> starts an OpenAI-compatible server with a built-in web UI.

vLLM is documented at docs.vllm.ai; the latest docs are marked developer preview, with a stable build under /en/stable/. It is the wrong tool for a double-click desktop app and the right one when several users share a GPU.

Pointing a frontend at it

SillyTavern speaks the KoboldAI API and OpenAI-compatible APIs, so all four projects can sit behind it. KoboldCpp serves http://localhost:5001, with an OpenAI-compatible endpoint at /v1 plus Anthropic and Ollama-compatible endpoints. Oobabooga exposes OpenAI/Anthropic-compatible Chat, Completions and Messages endpoints with tool-calling. llama.cpp's llama serve and vLLM both serve OpenAI-compatible APIs. For the frontend side, see our SillyTavern guide.

What it costs

All four are open-source and free to download and run: Oobabooga AGPL-3.0, KoboldCpp AGPL-3.0, llama.cpp MIT. vLLM's docs page does not restate a licence, so we are not naming one — check the repository. None charges per token; the recurring cost is hardware and electricity.

Optional cloud paths exist: KoboldCpp documents Colab and RunPod options at variable prices, which we are not quoting. Comparing local running with per-token APIs is a separate calculation; our SillyTavern guide covers the frontend-free/backend-paid split.

Two traps worth knowing

KoboldCpp's README warns: "koboldcpp.com is a malicious fake site and is not affiliated with this project." The official site is koboldcpp.net; downloads should come from GitHub releases.

Oobabooga's repository has moved: oobabooga/text-generation-webui is now oobabooga/textgen. Older tutorials still point at the former name.

The desk's take

Opinion, plainly: if you have never run a local model, start with KoboldCpp. One file, a documented five-step start and a UI that already understands characters and world info is the shortest path to a working roleplay backend. Pick Oobabooga if you want one app that switches between llama.cpp, ExLlamaV3 and Transformers and also does image generation. Pick llama.cpp if you are comfortable with a CLI and want the layer everything else is built on. Pick vLLM only if you are serving several users — it is infrastructure, not a desktop toy.

What we could not verify

  • No first-party VRAM-per-quantisation table exists across these projects; sizing stays model-, quant- and context-specific.
  • vLLM's licence is not restated on the docs page we read, so we did not name it.
  • No first-party tokens-per-second or benchmark numbers were recorded, so this guide contains none.