GPU Value Calculator

Choosing a graphics card (GPU) is one of the most important decisions you make when building a computer or upgrading your workstation. Here are a few key assessment dimensions:

① Determine the budget according to the purpose — Game-focused users should focus on FP32 performance and VRAM, while AI users should focus on VRAM and Tensor Core as core indicators. Video editing and 3D rendering fall somewhere in between, focusing on encoder support and CUDA core count.

② VRAM capacity is key — 8GB is enough for 1080p gaming, but AI inference (especially LLM) requires at least 12GB, with 24GB or more recommended. Stable Diffusion can smoothly generate 1024×1024 images above 12GB.

③ Pay attention to generational differences — Not only are new generation GPUs more efficient, they often also support newer technologies (such as DLSS 4, AV1 encoding, FP8/FP4 precision), which have a significant impact on AI and workloads.

VRAM (Video Random Access Memory) is GPU-specific memory used to store textures, buffers, AI model weights and intermediate calculation results required for rendering. VRAM size directly determines the size of the workload the GPU can handle.

In AI applications, VRAM is particularly critical: the weight parameters of a large language model (LLM) must be completely loaded into VRAM before it can run. For example, a 70B parameter model requires ~140GB VRAM at FP16, and even Q4 quantization requires ~35GB. When VRAM is insufficient, the model is forced to use system RAM (via offloading), resulting in a significant slowdown in inference speed.

General recommendations: Gaming 1080p → 8GB, 1440p → 12GB, 4K → 16GB+; Getting Started with AI → 12GB, Professional AI → 24GB+, LLM deployment → 48GB+ (or use multiple GPUs).

CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model launched by NVIDIA that allows developers to utilize a large number of GPU cores for general-purpose computing (GPGPU). CUDA cores are the basic computing units in NVIDIA GPUs, similar to CPU cores but in large numbers (RTX 4090 has 16384 CUDA cores).

In the field of AI, the CUDA ecosystem (with libraries such as cuDNN and TensorRT) has almost become an industry standard: PyTorch, TensorFlow, JAX and other frameworks all rely on CUDA acceleration at the bottom level. This means NVIDIA GPUs have a natural ecosystem advantage in AI workloads.

AMD uses Stream Processors to achieve similar functionality, paired with the ROCm platform. Although AMD has made significant progress in AI framework support in recent years, the maturity of the overall ecosystem still lags behind NVIDIA, especially in applications such as LLM inference and Stable Diffusion.

Tensor Core is a dedicated hardware unit in NVIDIA GPUs specifically designed to perform matrix multiplication and accumulation operations (GEMM), the most important type of operation in deep learning. Since the introduction of the Volta architecture (2017), Tensor Core has evolved through multiple generations, each supporting more precision formats.

The precisions supported by Tensor Core include: FP16 (16-bit floating point), BF16 (bfloat16), TF32 (Tensor Float 32), INT8 (8-bit integer), INT4 and FP4 (the latest Blackwell architecture). FP16 matrix operations using Tensor Cores are up to 8x faster than traditional FP32 (up to 16x with sparsification).

In addition to AI training and inference, Tensor Core is also used in NVIDIA DLSS (Deep Learning Super Sampling) technology, which uses AI to instantly reconstruct low-resolution images into high-resolution images, improving image quality without sacrificing performance. For AI users, the number and generations of Tensor Cores are important indicators for evaluating GPUs.

When building an AI workstation, GPU selection directly affects work efficiency and the size of the model that can be used. Here are some professional tips:

VRAM first — For AI, VRAM is the only hard limit. A model with a parameter size of 7B (such as Llama 3 8B) requires 16GB VRAM under FP16, and a 70B model requires about 140GB. It is recommended to have at least 24GB (RTX 3090/4090), and professional users can go up to 48GB or more (RTX 5090 or dual 4090).

Tensor Core Generations — The newer Tensor Core supports more precision formats (such as FP8, FP4), which can directly improve training speed and memory efficiency. The RTX 40 series (Ada Lovelace) is about 30-50% faster than the RTX 30 series (Ampere) in AI training.

Ecosystem compatibility — Currently, the AI framework has the most complete support for NVIDIA CUDA, and AMD ROCm is catching up, but compatibility still needs to be confirmed. For production environments, NVIDIA remains the safest choice. Consumer-grade GPUs (RTX) are more cost-effective than data center-grade (A100/H100) and are suitable for individuals and small teams.

The used GPU market offers excellent value for money, but it also comes with risks. The following is a comprehensive analysis:

Advantages: Prices are typically 50-70% of the original price (even lower for GPUs two generations ago). The second-hand price of RTX 3090 24GB is about NT$20,000-25,000, which is extremely cost-effective. For AI users, the VRAM advantages of older-generation flagships often make them more suitable than new-generation mid-range cards.

Disadvantages and risks: ① Mining card risks - GPUs that have been mining for a long time will continue to operate at full load, so the cooling modules are aging and the fans are seriously worn; ② Warranty issues - second-hand cards usually have no original warranty or have a short warranty period; ③ Generational technology differences - old GPUs lack new features (such as AV1 encoding, DLSS 4, FP8 support).

Buying advice: Give priority to sellers with original proof of purchase and avoid purchasing GPUs that are obviously used for mining (multiple cards without baffles and original boxes). RTX 3090, RTX 3080 Ti, and RTX 2080 Ti are currently the most cost-effective choices for AI applications in the second-hand market. The seller is required to provide GPU-Z screenshots and stress test results before transaction.