PromptMeterLLM SIZING & COST

Local VRAM · GGUF quantization · API batch cost

Will that model fit your GPU — and what will the batch cost?

PromptMeter answers two questions before you download a GGUF file or send an API batch: how much VRAM the model needs for its weights, KV cache and activation overhead at your quantization, context length and batch size, and what one run of your batch costs on a dated price table you can edit. Both numbers come with the formula behind them, the reference date of the data, and a plain note of what the estimate cannot see. Everything is computed in this tab — no upload, no account, no server call.

Pick a model preset from 1.5B to 671B or type your own parameters, choose FP16 down to Q2_K, set 2K–128K of context, FP16/Q8/Q4 KV cache and a batch size, and the page splits the total VRAM into weights, KV cache and overhead, marks each GPU capacity from 8 GB to 80 GB as fits or short, counts the cards a multi-GPU build needs, and gives a bandwidth-based tokens-per-second estimate. The second tab does the API side: context utilisation, per-request and batch cost, wall-clock time and a 30-day projection, against a price table whose basis date is on screen and whose every cell you can change.

Price basis 2026-11-21VRAM model weights + KV + overheadTool network calls 0

PromptMeter calculators

Model and runtime

Total VRAM needed

0GiB

—
  • Weights 0 GiB
  • KV cache 0 GiB
  • Overhead 0 GiB
KV per token—
VRAM use—
Cards needed—
Headroom—
Single stream—tok/s
Batch total—tok/s
Per request—tok/s

GPU fit at this setting

    How this total was calculated

    —

    weights_GiB = P × bpw ÷ 8 × 0.9313 KV_GiB = kv_KiB/token ÷ 1,048,576 × tokens × batch overhead_GiB = 0.40 + 0.10 × √P + 0.00025 × (tokens ÷ 1024) × batch tok/s ≈ bandwidth × 0.55 ÷ weights_GiB

    The per-token KV value is for this model's layer count and KV heads at FP16; the quantization factor and the batch size are applied on top.

    What it does

    Two calculators, one honest page

    Sizing on the left of the decision, spending on the right.

    VRAM

    One total, split into its parts

    Weights from parameters and bits per weight, KV cache from layers, context and batch, plus an activation allowance. You see the three parts and their share, not just a number.

    GGUF

    Seven quantization levels

    FP16, Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M and Q2_K, each with its approximate bits per weight, so the saving of one step down is visible before you download anything.

    KV

    Context and KV cache quantization

    Context from 2K to 128K, KV cache in FP16, Q8 or Q4, and batch size applied to every token stored. Long context and batching multiply the same cache.

    FIT

    GPU verdicts and card count

    Each capacity from 8 GB to 80 GB is marked fits or short in GiB, with a safety reserve you can switch off, and the selected card's count is computed for multi-GPU builds.

    SPEED

    Tokens per second, with the formula

    A memory-bandwidth approximation for single-stream and batched decode, so the number is checkable rather than mysterious — and it says how far it can be off.

    COST

    API batch cost you can trust or edit

    Context utilisation, per-request and batch cost, wall-clock and monthly projection, from a price table dated on screen; change any unit price and the result follows instantly.

    How it works

    From a model name to a decision in four steps

    No sign-up and no setup; every step happens on your device.

    01

    Pick the model

    Choose a preset or select Custom and type the parameter count and KV size per token. Both fields are shown so nothing is hidden in a preset.

    02

    Set the runtime shape

    Quantization, context length, KV cache precision and batch size decide most of the total. The breakdown and the GPU verdicts update as you type.

    03

    Read the fit

    The total, the three parts, the safety margin, the cards you need and the tokens/s estimate sit together, with the notes that apply to your current settings.

    04

    Price the API and export

    Switch to the API tab, set tokens and volume, adjust any unit price, then copy or download the whole configuration as JSON or Markdown.

    Reference

    GGUF quantization levels and what they weigh

    Weights only, in GiB, before any KV cache or overhead. The calculator uses the same bits-per-weight values.

    LevelBits / weight8B weights (GiB)70B weights (GiB)405B weights (GiB)
    FP16
    16-bit
    16.0014.90130.4754
    Q8_0
    8-bit, near-lossless
    8.507.9269.3401
    Q6_K
    K-quant
    6.566.1153.5309
    Q5_K_M
    K-quant, medium
    5.695.3046.4268
    Q4_K_M
    K-quant, medium
    4.854.5239.5229
    Q3_K_M
    K-quant, medium
    3.913.6431.9184
    Q2_K
    smallest, visibly lossy
    2.632.4521.4124

    Bits per weight are the usual averages reported for llama.cpp K-quants and can differ slightly between files. Q2_K and Q3_K_M trade visible quality for size; Q4_K_M is the common default for a reason.

    Reference

    GPU memory and bandwidth

    The last column applies the page's tokens/s formula to an 8B model at Q4_K_M (about 4.5 GiB of weights), single stream, before any KV cache.

    GPUVRAM (GB)Bandwidth (GB/s)8B Q4_K_M, ~tok/s
    RTX 4060 Ti 16 GB16288~35
    RTX 4070 Ti SUPER 16 GB16672~82
    RTX 4080 SUPER 16 GB16736~90
    RTX 3090 24 GB24936~114
    RTX 4090 24 GB241008~123
    RTX 5090 32 GB321792~218
    RTX A6000 48 GB48768~94
    RTX 6000 Ada 48 GB48960~117
    L40S 48 GB48864~105
    A100 80 GB802039~248
    H100 80 GB SXM803350~408
    Apple M4 Max 128 GB (unified)128546~66
    Apple M2 Ultra 192 GB (unified)192800~97

    Bandwidth figures are the manufacturers' peak memory bandwidth. Real decode throughput is usually 40–70% of the value this formula gives, depending on the runtime, the kernel and what else the GPU is doing.

    Method

    Exactly how the numbers are produced

    Every formula on this page is written out here, including the assumptions.

    Weights are the easy part: parameters × bits per weight ÷ 8, converted from billions of bytes to GiB. A 70B model at Q4_K_M is 70.6 × 4.85 ÷ 8 = 42.8 billion bytes, or 39.9 GiB. Nothing about the context or the batch changes this number, which is why it is the floor of every build.

    The KV cache is what people underestimate. It is two tensors (key and value) per layer, per token, per sequence, so its size grows with context length and batch size at the same time. For Llama 3.1 8B each token costs 128 KiB at FP16, which is 1 GiB at 8K context and 8 GiB at 64K. Quantizing the cache to Q8 roughly halves it, to Q4 roughly quarters it.

    Activation overhead covers the compute buffers, logits and fragmentation a runtime needs beyond weights and cache. It depends on the kernel and the batch, so this page uses a simple stated allowance: 0.40 GiB plus 0.10 GiB per square root of the parameter count in billions, plus 0.00025 GiB per 1,024 tokens per batch slot. Treat it as a planning allowance.

    The API side is arithmetic on published list prices: prompt tokens and completion tokens are priced separately per million, and the output side usually dominates. Timing is a simple queue model: a fixed first-token delay, prompt processing at an assumed prefill rate, output generation at an assumed decode rate, and requests grouped into waves by the concurrency you set. Providers' own latency varies with load, region and thinking tokens, so read the result as an order of magnitude, not a promise.

    Weights
    weights_GiB = P(B) × bpw ÷ 8 × 0.9313
    0.9313 converts 10⁹ bytes to GiB. P is in billions of parameters.
    KV cache
    KV_GiB = kv_KiB_per_token × factor ÷ 1,048,576 × tokens × batch
    factor is 1 for FP16, 0.53 for Q8 and 0.28 for Q4, including scale overhead.
    Activation overhead
    overhead_GiB = 0.40 + 0.10 × √P + 0.00025 × (tokens ÷ 1024) × batch
    A planning allowance, not a measurement.
    Total
    total_GiB = weights + KV + overhead
    Compared with capacity − 1 GiB − 4% when the safety reserve is on.
    Decode speed
    tok/s ≈ bandwidth_GB/s × 0.55 ÷ weights_GiB; batch × batch^0.72
    Bandwidth-bound decode with a 55% efficiency factor; prompt processing is not included.
    API cost
    per_request = prompt ÷ 1M × price_in + output ÷ 1M × price_out
    Batch cost is per request × requests; daily and 30-day figures use your daily volume.
    API time
    latency ≈ 0.6 s + prompt ÷ prefill + output ÷ decode; wall = ⌈requests ÷ concurrency⌉ × latency
    Prefill and decode rates are editable assumptions in the price table.

    Questions

    Frequently asked

    How much VRAM does an 8B model need at Q4_K_M?

    About 4.5 GiB for the weights (8 × 4.85 bits per weight), plus the KV cache and overhead. For Llama 3.1 8B at 8K context the cache is 1 GiB, and the page's overhead allowance is about 0.7 GiB, so the plan lands near 6.2 GiB. An 8 GB card fits with about 0.5 GiB of headroom after the safety reserve; at 32K context the cache alone is 4 GiB and the total passes 9 GiB, so the same card no longer fits.

    And a 70B model?

    At Q4_K_M the weights alone are about 39.9 GiB, the KV cache adds 2.5 GiB at 8K context and the overhead allowance about 1.2 GiB, so roughly 43.6 GiB. A 48 GB card holds it, and so do two 24 GB cards once the safety reserve is applied; tensor parallelism then costs some effective bandwidth.

    Why is the VRAM my machine reports higher than this number?

    The calculator counts weights, KV cache and a fixed activation allowance. A real run also holds the CUDA context, the graphics driver and desktop, allocator fragmentation, and runtime-specific buffers; compiled kernels and long-context attention can add more. Keep 10–20% free, which is what the safety margin is for.

    Does KV cache quantization really save that much?

    Roughly: Q8 about half and Q4 about a quarter of the FP16 cache, plus scale overhead. On Llama 3.1 8B at 64K context that turns 8 GiB into about 4.2 GiB or 2.2 GiB. It needs flash-attention style kernels and is not supported by every runtime, and quality can drop on long contexts, so measure before relying on it.

    Where do the API prices come from, and how current are they?

    They are list prices read from the providers' public price pages on 2026-11-21, shown in the table with that date. Some are promotional: Gemini 4 Carbon is 2/10 per million until 2026-11-30 and 4/20 after, Gemini 3.8 Flash is 0.75/3.75 until 2026-12-31, and DeepSeek prices halve off-peak. Every cell is editable, including context windows, so you can put in your own contract rate.

    How accurate is the tokens-per-second estimate?

    It is a bandwidth-bound approximation: memory bandwidth × 0.55 ÷ weight size in GiB. For an 8B Q4_K_M model on a card with 1,000 GB/s it gives about 120 tok/s, which is in the range people report for that combination. Compute-bound models, very small batches, CPU offload and prompt processing can all miss it by much more than 40%, so treat it as a comparison between setups.

    Is anything I type sent anywhere?

    No. Both calculators, the price table and the export are JavaScript in this page; the tool makes no fetch, XHR or beacon call. Your settings are kept in your own browser's localStorage and the only external request on the page is the advertising loader in the head.

    Honest limits

    What this page cannot tell you

    Estimates, not measurements

    The VRAM total is a model, not a reading from your machine. Different runtimes, kernels and compiled attention paths use different amounts, and a configuration that the page marks as fitting can still fail to load.

    Overhead is a formula

    Activation memory depends on the batch shape, the kernel and the context; the 0.40 + 0.10√P allowance is deliberately simple. Very long contexts can push attention buffers far above it.

    Tokens/s ignores compute limits

    The estimate assumes decode is limited by reading the weights from memory. A compute-bound model, a tiny batch, CPU offload or a busy desktop GPU breaks that assumption.

    API prices move

    The table is a dated snapshot of list prices. Providers tier by context length, cache hits, region and batch processing, and reasoning tokens are billed as output; none of that is modelled here. Put your own rate into the table for a real budget.

    What the numbers are based on

    • Quantization — bits per weight for llama.cpp GGUF K-quants: FP16 16.0, Q8_0 8.5, Q6_K 6.56, Q5_K_M 5.69, Q4_K_M 4.85, Q3_K_M 3.91, Q2_K 2.63
    • KV cache — two tensors per layer per token; the page's per-token values come from each preset's layer count and KV head size
    • GPU data — manufacturer peak VRAM and memory bandwidth; unified-memory Macs included
    • API prices — provider list prices read on 2026-11-21, editable in the page
    • Privacy — no upload, no account, no analytics; settings stay in localStorage