NVIDIA GeForce RTX 4090
Consumer
VRAM
24 GB
Memory
GDDR6X
Bandwidth
1008 GB/s
TDP
450W
Suitable Workloads
Smaller Language Models
Inference for 7B-13B parameter models
Key Highlights
- Built on Ada Lovelace architecture
- 16,384 compute cores for parallel processing
- PCIe form factor
Memory
VRAM24 GB
Memory TypeGDDR6X
Memory Bandwidth1008 GB/s
Compute Performance
FP32 (Single Precision)82.6 TFLOPS
FP16 (Half Precision)330 TFLOPS
BF16330 TFLOPS
Architecture
ArchitectureAda Lovelace
Compute Cores16,384
Release Year2022
Power & Physical
TDP450 W
Form FactorPCIe
PCIe GenerationPCIe 4.0
Notes
FP16 dense tensor with FP16 accumulate. The de facto self-host workhorse for quantized models up to ~32B.
Models That May Fit on NVIDIA GeForce RTX 4090
Estimates based on INT8 quantization. Actual fit depends on framework and batch size.
Explore More
Added Jul 30, 2026
Last updated: Jul 30, 2026
Your AI stack shouldn't stand still.
Every month new models become cheaper, faster, and more capable. Inferbase ensures your application automatically benefits without changing a single API call.