GPU Guide

LLM token generation is memory-bandwidth bound — more GB/s means faster inference. Sort by bandwidth, filter by VRAM and budget, and click any card for full specs to find the right GPU for running models locally.

144 GPUs

#	Card ↕	Generation ↕	VRAM ↕	Bandwidth ↓	Memory	Core / Boost
🥇	NRTX 5090	Blackwell	32 GB	1792 GB/s	512-bit GDDR7 · 28 Gbps	2017 / 2407 MHz
🥈	NRTX PRO 6000 Blackwell	Pro	96 GB	1792 GB/s	512-bit GDDR7 · 28 Gbps	1590 / 2617 MHz
🥉	NCMP 170HXno display	Ampere	8 GB	1493 GB/s	HBM2e 4096-bit	1140 / 1410 MHz
#4	ARadeon VIIAMD · ROCm	Vega	16 GB	1024 GB/s	HBM2 4096-bit	1400 / 1750 MHz
#5	NRTX 3090 Ti	Ampere	24 GB	1008 GB/s	384-bit GDDR6X · 21 Gbps	1560 / 1860 MHz
#6	NRTX 4090	Ada	24 GB	1008 GB/s	384-bit GDDR6X · 21 Gbps	2235 / 2520 MHz
#7	NRTX 5080	Blackwell	16 GB	960 GB/s	256-bit GDDR7 · 30 Gbps	2295 / 2617 MHz
#8	ARX 7900 XTXAMD · ROCm	RDNA3	24 GB	960 GB/s	384-bit GDDR6 · 20 Gbps	2300 / 2499 MHz
#9	NRTX 3090	Ampere	24 GB	936 GB/s	384-bit GDDR6X · 20 Gbps	1395 / 1695 MHz
#10	NRTX 3080 12GB	Ampere	12 GB	912 GB/s	384-bit GDDR6X · 19 Gbps	1260 / 1710 MHz
#11	NRTX 3080 Ti	Ampere	12 GB	912 GB/s	384-bit GDDR6X · 19 Gbps	1365 / 1665 MHz
#12	NRTX 5070 Ti	Blackwell	16 GB	896 GB/s	256-bit GDDR7 · 28 Gbps	2295 / 2452 MHz
#13	ARX 9070 XTAMD · ROCm	RDNA4	16 GB	896 GB/s	256-bit GDDR6 · 28 Gbps	2400 / 2970 MHz
#14	NRTX A6000 Ada	Pro	48 GB	864 GB/s	384-bit GDDR6 · 18 Gbps	915 / 2520 MHz
#15	ARadeon Pro W7900AMD · ROCm	RDNA3	48 GB	864 GB/s	384-bit GDDR6 · 18 Gbps	1900 / 2500 MHz
#16	ARX 7900 XTAMD · ROCm	RDNA3	20 GB	800 GB/s	320-bit GDDR6 · 20 Gbps	2000 / 2394 MHz
#17	NRTX A5000	Pro	24 GB	768 GB/s	384-bit GDDR6 · 16 Gbps	930 / 1695 MHz
#18	NRTX A6000	Pro	48 GB	768 GB/s	384-bit GDDR6 · 16 Gbps	930 / 1800 MHz
#19	NRTX 3080 10GB	Ampere	10 GB	760 GB/s	320-bit GDDR6X · 19 Gbps	1440 / 1710 MHz
#20	NRTX 4080	Ada	16 GB	736 GB/s	256-bit GDDR6X · 23 Gbps	2205 / 2505 MHz
#21	NRTX 4080 Super	Ada	16 GB	736 GB/s	256-bit GDDR6X · 23 Gbps	2295 / 2550 MHz
#22	NTITAN RTX	Turing	24 GB	672 GB/s	384-bit GDDR6 · 14 Gbps	1350 / 1770 MHz
#23	NRTX 4070 Ti Super	Ada	16 GB	672 GB/s	256-bit GDDR6X · 21 Gbps	2340 / 2610 MHz
#24	NRTX 5070	Blackwell	12 GB	672 GB/s	192-bit GDDR7 · 28 Gbps	2164 / 2512 MHz
#25	NQuadro RTX 6000	Turing	24 GB	672 GB/s	384-bit GDDR6 · 14 Gbps	1440 / 1770 MHz
#26	NQuadro RTX 8000	Turing	48 GB	672 GB/s	384-bit GDDR6 · 14 Gbps	1395 / 1770 MHz
#27	ARX 9070AMD · ROCm	RDNA4	16 GB	672 GB/s	256-bit GDDR6 · 21 Gbps	2070 / 2520 MHz
#28	NRTX A4500	Pro	20 GB	640 GB/s	320-bit GDDR6 · 16 Gbps	930 / 1650 MHz
#29	NRTX 2080 Ti	Turing	11 GB	616 GB/s	352-bit GDDR6 · 14 Gbps	1350 / 1545 MHz
#30	NRTX 3070 Ti	Ampere	8 GB	608 GB/s	256-bit GDDR6X · 19 Gbps	1575 / 1770 MHz
#31	NRTX 5000 Ada	Pro	16 GB	576 GB/s	256-bit GDDR6 · 18 Gbps	1425 / 2550 MHz
#32	ARX 7800 XTAMD · ROCm	RDNA3	16 GB	576 GB/s	256-bit GDDR6 · 18 Gbps	2124 / 2430 MHz
#33	ARX 7900 GREAMD · ROCm	RDNA3	16 GB	576 GB/s	256-bit GDDR6 · 18 Gbps	1880 / 2245 MHz
#34	ARadeon Pro W7800AMD · ROCm	RDNA3	32 GB	576 GB/s	256-bit GDDR6 · 18 Gbps	1800 / 2500 MHz
#35	ARX 6950 XTAMD · ROCm	RDNA2	16 GB	576 GB/s	256-bit GDDR6 · 18 Gbps	2100 / 2310 MHz
#36	iArc A770 8GBIntel · oneAPI	Arc	8 GB	560 GB/s	256-bit GDDR6 · 18 Gbps	2100 / 2400 MHz
#37	iArc A770 16GBIntel · oneAPI	Arc	16 GB	560 GB/s	256-bit GDDR6 · 18 Gbps	2100 / 2400 MHz
#38	NTITAN Xp	Pascal	12 GB	548 GB/s	384-bit GDDR5X · 11 Gbps	1405 / 1582 MHz
#39	ARX 6800AMD · ROCm	RDNA2	16 GB	512 GB/s	256-bit GDDR6 · 16 Gbps	1815 / 2105 MHz
#40	ARX 6800 XTAMD · ROCm	RDNA2	16 GB	512 GB/s	256-bit GDDR6 · 16 Gbps	2015 / 2250 MHz
#41	ARX 6900 XTAMD · ROCm	RDNA2	16 GB	512 GB/s	256-bit GDDR6 · 16 Gbps	2015 / 2250 MHz
#42	ARadeon Pro W6800AMD · ROCm	RDNA2	32 GB	512 GB/s	256-bit GDDR6 · 16 Gbps	1700 / 2095 MHz
#43	NRTX 4070	Ada	12 GB	504 GB/s	192-bit GDDR6X · 21 Gbps	1920 / 2475 MHz
#44	NRTX 4070 Super	Ada	12 GB	504 GB/s	192-bit GDDR6X · 21 Gbps	1980 / 2475 MHz
#45	NRTX 4070 Ti	Ada	12 GB	504 GB/s	192-bit GDDR6X · 21 Gbps	2310 / 2610 MHz
#46	NRTX 2080 Super	Turing	8 GB	496 GB/s	256-bit GDDR6 · 16 Gbps	1650 / 1815 MHz
#47	NGTX 1080 Ti	Pascal	11 GB	484 GB/s	352-bit GDDR5X · 11 Gbps	1480 / 1582 MHz
#48	ARX Vega 64AMD · ROCm	Vega	8 GB	484 GB/s	HBM2 2048-bit	1274 / 1546 MHz
#49	NTITAN X (Pascal)	Pascal	12 GB	480 GB/s	384-bit GDDR5X · 10 Gbps	1417 / 1531 MHz
#50	iArc B580 12GBIntel · oneAPI	Arc	12 GB	456 GB/s	192-bit GDDR6 · 19 Gbps	2170 / 2670 MHz
#51	NRTX 2060 Super	Turing	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1470 / 1650 MHz
#52	NRTX 2070	Turing	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1410 / 1620 MHz
#53	NRTX 2070 Super	Turing	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1605 / 1770 MHz
#54	NRTX 2080	Turing	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1515 / 1710 MHz
#55	NRTX 3060 Ti	Ampere	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1410 / 1665 MHz
#56	NRTX 3070	Ampere	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1500 / 1725 MHz
#57	NRTX 5060narrow bus	Blackwell	8 GB	448 GB/s	128-bit GDDR7 · 28 Gbps	2280 / 2497 MHz
#58	NRTX 5060 Ti 8GBnarrow bus	Blackwell	8 GB	448 GB/s	128-bit GDDR7 · 28 Gbps	2377 / 2572 MHz
#59	NRTX 5060 Ti 16GBnarrow bus	Blackwell	16 GB	448 GB/s	128-bit GDDR7 · 28 Gbps	2377 / 2572 MHz
#60	NQuadro RTX 5000	Turing	16 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1620 / 1815 MHz
#61	NRTX A4000	Pro	16 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	735 / 1560 MHz
#62	ARX 5700AMD · ROCm	RDNA1	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1465 / 1725 MHz
#63	ARX 5700 XTAMD · ROCm	RDNA1	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1605 / 1905 MHz
#64	ARadeon Pro W5700AMD · ROCm	RDNA1	8 GB	448 GB/s	256-bit GDDR6 · 14 Gbps	1688 / 1860 MHz
#65	NP102-100 10GBno display	Pascal	10 GB	440 GB/s	320-bit GDDR5X · 11 Gbps	1442 / 1582 MHz
#66	NQuadro P6000	Pascal	24 GB	432 GB/s	384-bit GDDR5X · 9 Gbps	1506 / 1645 MHz
#67	NRTX 4000 Ada	Pro	20 GB	432 GB/s	192-bit GDDR6 · 18 Gbps	1500 / 2175 MHz
#68	NRTX 4500 Ada	Pro	24 GB	432 GB/s	192-bit GDDR6 · 18 Gbps	1800 / 2580 MHz
#69	ARX 7700 XTAMD · ROCm	RDNA3	12 GB	432 GB/s	192-bit GDDR6 · 18 Gbps	2171 / 2599 MHz
#70	ARX 6750 XTAMD · ROCm	RDNA2	12 GB	432 GB/s	192-bit GDDR6 · 18 Gbps	2321 / 2600 MHz
#71	NQuadro RTX 4000	Turing	8 GB	416 GB/s	256-bit GDDR6 · 13 Gbps	1005 / 1545 MHz
#72	ARX Vega 56AMD · ROCm	Vega	8 GB	410 GB/s	HBM2 2048-bit	1156 / 1471 MHz
#73	ARX 6700 XTAMD · ROCm	RDNA2	12 GB	384 GB/s	192-bit GDDR6 · 16 Gbps	2321 / 2581 MHz
#74	iArc A750Intel · oneAPI	Arc	8 GB	384 GB/s	192-bit GDDR6 · 16 Gbps	2050 / 2400 MHz
#75	iArc B570 10GBIntel · oneAPI	Arc	10 GB	380 GB/s	160-bit GDDR6 · 19 Gbps	2500 / 2750 MHz
#76	NRTX 3060	Ampere	12 GB	360 GB/s	192-bit GDDR6 · 15 Gbps	1320 / 1777 MHz
#77	NGTX 780 Ti	Kepler	3 GB	336 GB/s	384-bit GDDR5 · 7 Gbps	875 / 928 MHz
#78	NGTX 980 Ti	Maxwell	6 GB	336 GB/s	384-bit GDDR5 · 7 Gbps	1000 / 1076 MHz
#79	NTITAN X	Maxwell	12 GB	336 GB/s	384-bit GDDR5 · 7 Gbps	1000 / 1089 MHz
#80	NGTX 1660 Super	Turing	6 GB	336 GB/s	192-bit GDDR6 · 14 Gbps	1530 / 1785 MHz
#81	NRTX 2060	Turing	6 GB	336 GB/s	192-bit GDDR6 · 14 Gbps	1365 / 1680 MHz
#82	NRTX 2060 12GB	Turing	12 GB	336 GB/s	192-bit GDDR6 · 14 Gbps	1470 / 1650 MHz
#83	NRTX A3000 12GB	Pro	12 GB	336 GB/s	192-bit GDDR6 · 14 Gbps	1005 / 1560 MHz
#84	ARX 5600 XTAMD · ROCm	RDNA1	6 GB	336 GB/s	192-bit GDDR6 · 14 Gbps	1375 / 1620 MHz
#85	NGTX 1080	Pascal	8 GB	320 GB/s	256-bit GDDR5X · 10 Gbps	1607 / 1733 MHz
#86	NRTX 5050narrow bus	Blackwell	8 GB	320 GB/s	128-bit GDDR6 · 20 Gbps	2317 / 2572 MHz
#87	NRTX 4000 SFF Ada	Pro	20 GB	320 GB/s	160-bit GDDR6 · 16 Gbps	1350 / 1800 MHz
#88	ARX 6700AMD · ROCm	RDNA2	10 GB	320 GB/s	160-bit GDDR6 · 16 Gbps	2321 / 2424 MHz
#89	NQuadro M6000	Maxwell	24 GB	317 GB/s	384-bit GDDR5 · 7 Gbps	988 / 988 MHz
#90	NGTX 780	Kepler	3 GB	288 GB/s	384-bit GDDR5 · 6 Gbps	863 / 900 MHz
#91	NQuadro P5000	Pascal	16 GB	288 GB/s	256-bit GDDR5X · 9 Gbps	1607 / 1733 MHz
#92	NGTX 1660 Ti	Turing	6 GB	288 GB/s	192-bit GDDR6 · 12 Gbps	1500 / 1770 MHz
#93	NQuadro RTX 3000	Turing	6 GB	288 GB/s	192-bit GDDR6 · 12 Gbps	1620 / 1620 MHz
#94	NRTX 4060 Ti 8GB	Ada	8 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	2310 / 2535 MHz
#95	NRTX 4060 Ti 16GB	Ada	16 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	2310 / 2535 MHz
#96	NRTX A2000 12GB	Pro	12 GB	288 GB/s	192-bit GDDR6 · 12 Gbps	735 / 1200 MHz
#97	ARX 9060 XT 8GBAMD · ROCm	RDNA4	8 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	2200 / 2700 MHz
#98	ARX 9060 XT 16GBAMD · ROCm	RDNA4	16 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	2200 / 2700 MHz
#99	ARX 7600AMD · ROCm	RDNA3	8 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	2250 / 2655 MHz
#100	ARX 7600 XT 16GBAMD · ROCm	RDNA3	16 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	2310 / 2755 MHz
#101	iArc A580Intel · oneAPI	Arc	8 GB	288 GB/s	128-bit GDDR6 · 18 Gbps	1700 / 2000 MHz
#102	ARX 6650 XTAMD · ROCm	RDNA2	8 GB	280 GB/s	128-bit GDDR6 · 18 Gbps	2055 / 2635 MHz
#103	NRTX 4060	Ada	8 GB	272 GB/s	128-bit GDDR6 · 17 Gbps	1830 / 2460 MHz
#104	NGTX 1070	Pascal	8 GB	256 GB/s	256-bit GDDR5 · 8 Gbps	1506 / 1683 MHz
#105	NGTX 1070 Ti	Pascal	8 GB	256 GB/s	256-bit GDDR5 · 8 Gbps	1607 / 1683 MHz
#106	ARX 6600 XTAMD · ROCm	RDNA2	8 GB	256 GB/s	128-bit GDDR6 · 16 Gbps	2359 / 2589 MHz
#107	ARX 580 8GBAMD · ROCm	Polaris	8 GB	256 GB/s	256-bit GDDR5 · 8 Gbps	1257 / 1340 MHz
#108	ARX 590 8GBAMD · ROCm	Polaris	8 GB	256 GB/s	256-bit GDDR5 · 8 Gbps	1469 / 1545 MHz
#109	NQuadro P4000	Pascal	8 GB	243 GB/s	256-bit GDDR5 · 8 Gbps	1202 / 1480 MHz
#110	NRTX 3060 8GBnarrow bus	Ampere	8 GB	240 GB/s	128-bit GDDR6 · 15 Gbps	1320 / 1777 MHz
#111	NGTX 770	Kepler	2 GB	224 GB/s	256-bit GDDR5 · 7 Gbps	1046 / 1085 MHz
#112	NGTX 970	Maxwell	4 GB	224 GB/s	256-bit GDDR5 · 7 Gbps	1050 / 1178 MHz
#113	NGTX 980	Maxwell	4 GB	224 GB/s	256-bit GDDR5 · 7 Gbps	1126 / 1216 MHz
#114	NRTX 3050 8GB	Ampere	8 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	1552 / 1777 MHz
#115	NRTX 3050 16GB	Ampere	16 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	1042 / 1470 MHz
#116	NRTX 2000 Ada	Pro	16 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	405 / 2175 MHz
#117	ARX 6600AMD · ROCm	RDNA2	8 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	1626 / 2044 MHz
#118	ARadeon Pro W6600AMD · ROCm	RDNA2	8 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	1589 / 1920 MHz
#119	ARX 5500 XT 4GBAMD · ROCm	RDNA1	4 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	1607 / 1845 MHz
#120	ARX 5500 XT 8GBAMD · ROCm	RDNA1	8 GB	224 GB/s	128-bit GDDR6 · 14 Gbps	1607 / 1845 MHz
#121	ARX 570 8GBAMD · ROCm	Polaris	8 GB	224 GB/s	256-bit GDDR5 · 7 Gbps	1168 / 1244 MHz
#122	NQuadro M5000	Maxwell	8 GB	211 GB/s	256-bit GDDR5 · 7 Gbps	1051 / 1051 MHz
#123	NGTX 760	Kepler	2 GB	192 GB/s	256-bit GDDR5 · 6 Gbps	980 / 1033 MHz
#124	NQuadro M4000	Maxwell	8 GB	192 GB/s	256-bit GDDR5 · 6 Gbps	773 / 773 MHz
#125	NGTX 1060 3GB	Pascal	3 GB	192 GB/s	192-bit GDDR5 · 8 Gbps	1506 / 1709 MHz
#126	NGTX 1060 6GB	Pascal	6 GB	192 GB/s	192-bit GDDR5 · 8 Gbps	1506 / 1709 MHz
#127	NGTX 1650 Super	Turing	4 GB	192 GB/s	128-bit GDDR6 · 12 Gbps	1530 / 1725 MHz
#128	NGTX 1660	Turing	6 GB	192 GB/s	192-bit GDDR5 · 8 Gbps	1530 / 1785 MHz
#129	NRTX A2000 6GB	Pro	6 GB	192 GB/s	128-bit GDDR6 · 12 Gbps	735 / 1200 MHz
#130	ARX 7500 XTAMD · ROCm	RDNA3	8 GB	192 GB/s	128-bit GDDR6 · 12 Gbps	2254 / 2609 MHz
#131	iArc A380Intel · oneAPI	Arc	6 GB	186 GB/s	96-bit GDDR6 · 16 Gbps	2000 / 2450 MHz
#132	NRTX 3050 6GBnarrow bus	Ampere	6 GB	168 GB/s	96-bit GDDR6 · 14 Gbps	1552 / 1777 MHz
#133	ARX 6500 XTAMD · ROCmnarrow bus	RDNA2	4 GB	144 GB/s	64-bit GDDR6 · 18 Gbps	2610 / 2815 MHz
#134	NGTX 1650	Turing	4 GB	128 GB/s	128-bit GDDR5 · 8 Gbps	1485 / 1665 MHz
#135	ARX 6400AMD · ROCmnarrow bus	RDNA2	4 GB	128 GB/s	64-bit GDDR6 · 16 Gbps	1923 / 2321 MHz
#136	NGTX 960	Maxwell	4 GB	112 GB/s	128-bit GDDR5 · 7 Gbps	1127 / 1178 MHz
#137	NGTX 1050	Pascal	2 GB	112 GB/s	128-bit GDDR5 · 7 Gbps	1354 / 1455 MHz
#138	NGTX 1050 Ti	Pascal	4 GB	112 GB/s	128-bit GDDR5 · 7 Gbps	1290 / 1392 MHz
#139	NGTX 950	Maxwell	2 GB	105 GB/s	128-bit GDDR5 · 7 Gbps	1024 / 1188 MHz
#140	NGTX 1630narrow bus	Turing	4 GB	96 GB/s	64-bit GDDR6 · 12 Gbps	1785 / 1800 MHz
#141	iArc A310Intel · oneAPInarrow bus	Arc	4 GB	96 GB/s	64-bit GDDR6 · 12 Gbps	1550 / 2000 MHz
#142	NGTX 750 Ti	Maxwell	2 GB	86 GB/s	128-bit GDDR5 · 5 Gbps	1020 / 1085 MHz
#143	NGTX 750	Maxwell	2 GB	80 GB/s	128-bit GDDR5 · 5 Gbps	1020 / 1085 MHz
#144	NGTX 1030narrow bus	Pascal	2 GB	48 GB/s	64-bit GDDR5 · 6 Gbps	1227 / 1468 MHz