🟢 All systems operational · Apache-2.0 serving engine on hardware you rent
+ Add Provider
Apache-2.0 · production serving

vLLM — Built for
Many Requests at Once

vLLM calls itself a high-throughput, memory-efficient inference and serving engine, and the emphasis on memory is the interesting half. It started in the Sky Computing Lab at UC Berkeley and now carries contributions from more than two thousand people. You reach for it when one model has to answer a lot of people at the same time.

See what a GPU costs → Browse all GPU plans
vllm — serve
$ uv pip install vllm
$ vllm serve <model> \
    --tensor-parallel-size 2 \
    --gpu-memory-utilization 0.90
✓ OpenAI-compatible API on :8000
# the two flags above are the whole game:
# how many cards, and how much of each

What vLLM brings to the box

PagedAttention
The KV cache is managed in pages rather than one contiguous block per request, the way an operating system handles virtual memory. That is what stops half the card being lost to fragmentation when sessions have wildly different lengths.
Continuous batching
New requests join the batch already in flight instead of waiting for it to finish, with chunked prefill and prefix caching alongside. A shared system prompt gets computed once rather than once per user.
Quantization, most formats
FP8, MXFP8 and MXFP4, NVFP4, INT8 and INT4, GPTQ and AWQ, GGUF, compressed-tensors, ModelOpt and TorchAO. Whichever way the checkpoint you want was compressed, it probably loads.
Five kinds of parallelism
Tensor, pipeline, data, expert and context parallelism for distributed inference. This is how a model that will not fit one card gets served across several, and why the card count matters as much as the memory on each.
An OpenAI-shaped API, and more
An OpenAI-compatible server, plus the Anthropic Messages API and gRPC. Client code that already talks to a hosted model usually needs a base URL changed and nothing else.
Not only NVIDIA
NVIDIA and AMD GPUs, x86, ARM and PowerPC CPUs, plus Google TPUs, Intel Gaudi, IBM Spyre, Huawei Ascend, Rebellions NPU, Apple Silicon and MetaX. Useful when the cheapest card in a region is not the one everybody assumes.

Sizing for concurrency, not just for weights

This arithmetic is ours, not a vendor requirement. vLLM publishes no per-model GPU minimum, so we do the sum in the open: weights take roughly 2 bytes per parameter at FP16, 1 byte at FP8 and 0.5 byte at 4-bit, plus +25% for the KV cache and the serving runtime. It is a floor. On this page the floor is the easy part. vLLM pre-allocates whatever share of the card you allow it (--gpu-memory-utilization, 0.90 by default) and spends everything above the weights on the KV cache. That leftover is what decides how many people can talk to the model at once, and how long each conversation may get before something has to be evicted. Buying a card that exactly fits the weights gives you a model that serves one user.
Card memory Model size at FP16 Model size at 4-bit What the leftover buys
16 GB up to ≈ 6B up to ≈ 24B A handful of short sessions
24 GB 7–8B 30–32B A real batch at a modest context
48 GB 13–14B 70B Production concurrency on a mid-sized model
80 GB 30–32B 120B mixture-of-experts Long contexts, or many short ones
141 GB ≈ 50B Several models resident Long contexts and high concurrency

The middle two columns are ours, straight out of the rule above: card memory divided by bytes-per-parameter, then by the 1.25 headroom factor. They are the size that leaves the model room to breathe, not the largest thing that will technically load. The right-hand column is deliberately vague and stays that way: the KV cache per token depends on the model’s layer count, its attention layout and its quantization, so anyone quoting you a single “sessions per card” number without naming the model is guessing. Measure it with the checkpoint you actually plan to serve.

Cheapest GPU at each tier

All GPU plans →

The cheapest live GPU plan in the DohoHub catalogue at each memory tier, out of 142 plans whose provider publishes a GPU memory figure. Prices come from our own feed, normalised so they are comparable and shown in the currency picked in the header. Where one machine is the cheapest way to clear several tiers, it appears once and the note says which tiers it covers.

Provider Plan Specs Price Visit
Servarica
Bee GPU VPS
cheapest at 16–24 GB+ · 24 GB on board
 6 Cores24 GB vRAM30 GB RAM400 GB NVME100 TB traffic10 Gbps
$79.00/mo
 Visit
NovoServe
Supermicro X11 10SFF (GPU)
cheapest at 48 GB+ · 64 GB on board
 64 GB vRAM25 TB traffic
$355.99/mo
renews at $402.52
 Visit
GPU Mart
Enterprise GPU VPS - RTX Pro 6000
cheapest at 80 GB+ · 96 GB on board
 32 Cores96 GB vRAM84 GB RAM400 GB SSDUnlimited traffic1 Gbps
$649.00/mo
 Visit
Contabo
Dedicated NVIDIA H200
cheapest at 141 GB+ · 141 GB on board
 32 Cores141 GB vRAM234 GB RAM1900 GB15 TB traffic
$2,138.25/mo
 Visit

One caveat that bites harder on this page than the others: we record the memory a plan advertises, not how it is divided. vLLM will serve a model across several cards with tensor parallelism, but the interconnect between them then sets the ceiling, and no provider in our catalogue publishes that figure in a form we can compare. If the plan is multi-card, ask them before you assume. None of these providers sells “vLLM hosting”; they sell GPU servers, and the install is two commands.

Pick a card with room above the weights

Filter the catalogue by GPU memory, cores, storage and location — live prices, tracked every six hours.

Find a GPU server →
↑