NVIDIA T4
The T4 is NVIDIA's most widely deployed inference GPU with 16GB GDDR6 and 70W TDP. A cost-effective choice for inference at scale, available across all major cloud providers.
VRAM
16 GB
Memory
GDDR6
Bandwidth
320 GB/s
FP16
65 TF
TDP
70 W
Memory and compute
Architecture and power
Notes
Widely deployed for inference
Models that may fit on NVIDIA T4
Estimates based on INT8 quantization. Actual fit depends on framework and batch size.
More GPUs by NVIDIA
The manufacturer's other cards in the catalog, largest memory first, with bandwidth and power where published.
Frequently asked questions about NVIDIA T4
Answered from the entry's own figures: memory, which models fit, bandwidth, power and the card's class.
NVIDIA T4 has 16 GB of GDDR6 at 320 GB/s. At INT8 a model's weights take about one byte per parameter, so the card holds roughly a 12B-parameter model with room for context; larger models need several cards or a lower precision.
From the catalog's estimates at INT8, GLM Z1 9B 0414, VibeVoice ASR HF, Qwen 3 VL 8B Instruct and UI TARS 7B SFT fit on a single NVIDIA T4 with their default context. The list on this page shows the estimated memory each takes; the capacity planner sizes any model against this card for your own context length, batch size and request rate.
320 GB/s. Bandwidth bounds how fast a model generates tokens, because every token reads the whole set of weights and the KV cache from memory; at this rate a 12B model at INT8 could be read on the order of 25 times a second on one card, before batching and framework efficiency.
NVIDIA T4 has a TDP of 70 W, with a maximum of 70 W, in the PCIe Low-profile form factor. Budget the TDP per card plus host overhead when sizing a server's power and cooling.
NVIDIA T4 is a datacenter accelerator, meant for servers and multi-GPU serving, built on the Turing architecture and released in 2018. It has no multi-GPU interconnect, so a model must fit on one card or be split over the slower PCIe bus.
Specifications from the manufacturer's published material; a figure that is not published is shown as such, and a derived one is explained in the notes.
Added Jan 25, 2026 · Last updated Jan 25, 2026
Explore more
Companion tools that draw on the same catalog and routing engine.
Put intelligence in the middle.
One gateway in front of every model, with your policies applied and every decision on record. Start on the free plan with 5,000 requests a month, no card required.