Showing 39 of 39 GPUs
GPU · Catalog
GPU Catalog
Datacenter, workstation and consumer GPUs compared on memory, bandwidth, compute and power, with the models each one can hold.
The catalog
What a GPU entry carries
The figures a sizing decision turns on, as the manufacturer published them, and the models the catalog estimates would fit.
This GPU catalog covers the cards used for AI inference and training: datacenter accelerators in SXM and OAM packages, workstation boards, and the consumer line, from NVIDIA, AMD and the other manufacturers shipping AI silicon. Every entry is attributed to its manufacturer and typed by where it is meant to run.
Each entry shows memory size and type, memory bandwidth, compute at FP32, FP16, BF16 and INT8, the interconnect between cards, the architecture and package, and power, so you can compare GPUs for AI on the figures that bound a serving deployment. A figure the manufacturer has not published says so.
Open any entry for the models estimated to fit on it, its notes on how a figure was derived, and the manufacturer's other cards; the capacity planner sizes any model against it.
- Manufacturer
- AMD
- Type
- datacenter
- VRAM
- 432 GB HBM4
- Bandwidth
- 19,600 GB/s
- FP16
- 10,000 TFLOPS
- Interconnect
- multi-GPU, bandwidth not published
- TDP
- not published
Open the full entry: every precision, the architecture and package, the notes, and the models estimated to fit.
More tools
Explore more
Companion tools that draw on the same catalog and routing engine.
FAQ
Frequently asked questions about GPUs for AI
How much memory a model needs, which figure matters for what, and where the numbers come from.
A catalog of the GPUs used for AI inference and training: datacenter, workstation and consumer cards from NVIDIA, AMD and other manufacturers. Each entry records memory size and type, memory bandwidth, compute at each precision, interconnect, architecture, package and power, as the manufacturer published them, plus the models the catalog estimates would fit on the card.
Enough to hold the weights at the precision you serve, plus the working memory for the context and concurrency you run. At INT8 the weights take about one byte per parameter, so a 70B model needs on the order of 70 GB before context. The GPU Capacity Planner does the full arithmetic for a model, a quantization, a context length and a request rate, and each GPU entry lists the models estimated to fit at INT8.
The one whose memory holds your model with room for context, and whose memory bandwidth keeps token generation fast, since decoding is bandwidth-bound. Compute at FP16 or INT8 matters more for prompt processing and batching. Sort the catalog by the figure that limits you, then size the shortlist in the planner against your traffic.
Datacenter cards ship in SXM or OAM packages with high-bandwidth memory and fast inter-card links for multi-GPU serving. Workstation cards are PCIe boards with large memory for a single machine. Consumer cards are the gaming line, with less memory and no inter-card link, but the lowest price per card. The Type filter separates the three.
Yes. The filters set a minimum VRAM, a minimum FP16 compute, a minimum memory bandwidth and a maximum TDP, and narrow by manufacturer and type. The quick filters jump to high-memory and datacenter cards, and the table and card views show the same list.
Every GPU entry lists the models the catalog estimates would fit on it at INT8, and every model entry lists the GPUs that hold it, with a count above one where the weights need several cards. The estimate covers the weights and a default context; the planner refines it for your context length and batch size.
From the manufacturers’ published material. A figure that is not published is shown as not published rather than filled in, and where a figure is derived, such as a dense FP16 rate taken from a published FP8 rate, the entry’s notes say how.
No. The catalog and the capacity planner are for sizing what you host yourself. If you would rather not run GPUs, the same models are served through the gateway on managed capacity, priced per token.
Put intelligence in the middle.
One gateway in front of every model, with your policies applied and every decision on record. Start on the free plan with 5,000 requests a month, no card required.