A graphics processing unit (GPU) is a specialized electronic circuit originally designed to accelerate the rendering of images, animations, and video for computer displays. Modern GPUs are highly parallel processors that excel at performing simultaneous mathematical operations, making them essential not only for gaming and visual computing but also for general‑purpose computing tasks such as machine learning, scientific simulations, and cryptocurrency mining. The evolution of GPUs has transformed them from fixed‑function graphics accelerators into programmable, massively parallel processors that have become a cornerstone of modern computing.
1 Architecture
1.1 Core Components
The architecture of a modern GPU is built around an array of parallel processing units, a hierarchical memory system, and dedicated hardware for graphics‑specific tasks.
1.1.1 Streaming Multiprocessors (SMs) / Compute Units
The fundamental building blocks of a GPU are its streaming multiprocessors (SMs, in NVIDIA terminology) or compute units (CUs, in AMD terminology). Each SM or CU contains multiple arithmetic logic units (ALUs), also called CUDA cores or stream processors, along with control logic, schedulers, and local memory. These units execute thousands of threads concurrently, sharing instruction fetch and control hardware to maximize efficiency.
1.1.2 Memory Hierarchy
GPU memory is organized into a hierarchy that balances capacity, latency, and bandwidth to meet the demands of parallel workloads.
1.1.2.1 Global Memory
Global memory is the largest and slowest tier, typically implemented as dedicated GDDR (Graphics Double Data Rate) or HBM (High‑Bandwidth Memory) on discrete GPUs. It is accessible by all threads but requires explicit management for performance optimization. Global memory stores textures, vertex data, and intermediate results.
1.1.2.2 Shared Memory
Shared memory is a small, low‑latency memory located on each SM or CU. It is shared among threads within a thread block (or workgroup) and serves as a programmer‑managed cache for frequently accessed data, reducing reliance on slower global memory.
1.1.2.3 Registers and Local Memory
Registers are the fastest memory, allocated per thread for storing local variables. When register pressure exceeds available capacity, data is automatically spilled to local memory, which resides in global memory but is cached. Efficient register usage is critical for performance.
1.2 Pipeline Stages
The traditional graphics pipeline in a GPU consists of several sequential stages that transform 3D geometry into a 2D image.
1.2.1 Vertex Processing
In vertex processing, the GPU processes each vertex of a 3D model, applying transformations (such as model‑view‑projection), per‑vertex lighting, and tessellation. Vertex shaders, programmable by the user, control this stage.
1.2.2 Rasterization
Rasterization converts the transformed geometry (triangles, lines, points) into fragments (potential pixels). The rasterizer determines which fragments lie inside a primitive, interpolates vertex attributes (e.g., color, texture coordinates) across the triangle, and generates a stream of fragments for further processing.
1.2.3 Fragment Processing
Fragment processing, or pixel shading, calculates the final color of each fragment by applying textures, lighting calculations, and any programmable effects (e.g., blending, transparency). The output is written to the framebuffer, which is later displayed.
1.3 Execution Model
GPU execution is centered around massive parallelism, coordinated through a specific threading model.
1.3.1 Single Instruction, Multiple Thread (SIMT)
SIMT is an execution model where multiple threads execute the same instruction on different data elements. This differs from SIMD (Single Instruction, Multiple Data) in that each thread can have its own program counter and may diverge, though divergence reduces efficiency. SIMT enables fine‑grained parallelism and programming flexibility.
1.3.2 Warps/Wavefronts
Threads are grouped into batches for scheduling: a warp (NVIDIA, typically 32 threads) or a wavefront (AMD, typically 64 threads). The GPU schedules these groups onto SMs/CUs, issuing a single instruction to all threads in a warp. Threads that diverge due to conditional branches are serialized, lowering performance.
2 Types of GPUs
2.1 Integrated GPUs
Integrated GPUs (iGPUs) are built into the same die as the central processing unit (CPU) or packaged together in a system‑on‑chip (SoC). They share system memory and are designed for lower power consumption and cost, making them ideal for laptops, tablets, and budget desktops. Performance is sufficient for everyday computing and light gaming.
2.2 Discrete GPUs
Discrete GPUs are separate expansion cards connected via PCI Express (PCIe) to the motherboard. They have their own dedicated video memory (VRAM) and cooling solutions, offering far greater performance than integrated solutions. Discrete GPUs are the standard for gaming, professional visualization, and high‑performance computing.
2.3 Professional vs. Consumer GPUs
2.3.1 Workstation/Pro Cards
Workstation GPUs (e.g., NVIDIA Quadro/RTX A-series, AMD Radeon Pro) are certified for professional applications such as CAD, 3D modeling, and scientific visualization. They provide higher precision (e.g., double‑precision floating‑point), larger memory configurations, error‑correcting code (ECC) memory, and validated drivers for stable operation under demanding workloads.
2.3.2 Gaming Cards
Consumer gaming GPUs (e.g., NVIDIA GeForce, AMD Radeon) are optimized for real‑time rendering with high frame rates. They prioritize single‑precision performance, support for graphics APIs like DirectX and Vulkan, and features such as ray tracing and variable‑rate shading. They are also widely used for general‑purpose computing tasks.
3 Programming and APIs
3.1 Graphics APIs
Graphics APIs provide a standardized interface for applications to communicate with the GPU for rendering tasks.
3.1.1 DirectX
DirectX is a collection of Microsoft’s APIs, including Direct3D for 3D graphics. It is heavily used in Windows gaming and offers features like ray tracing (DirectX Raytracing, DXR), mesh shaders, and variable‑rate shading. Versions are tied to Windows and Xbox platforms.
3.1.2 Vulkan
Vulkan is a low‑overhead, cross‑platform graphics and compute API developed by the Khronos Group. It provides fine‑grained control over GPU resources and command buffering, reducing CPU overhead and enabling high performance on a wide range of devices, from mobile to desktop.
3.1.3 OpenGL
OpenGL is an older, cross‑platform API that is simpler but less efficient than Vulkan. It has been largely superseded by Vulkan for new development, though it remains in use for legacy applications and some scientific visualization.
3.2 General‑Purpose Computing APIs
General‑purpose GPU (GPGPU) APIs allow non‑graphics workloads to harness the parallel power of the GPU.
3.2.1 CUDA
CUDA (Compute Unified Device Architecture) is NVIDIA’s proprietary parallel computing platform and programming model. It extends C/C++ with syntax for defining kernels that run on the GPU, supporting thousands of threads. CUDA is widely adopted in scientific computing, machine learning, and deep learning frameworks.
3.2.2 OpenCL
OpenCL (Open Computing Language) is an open, cross‑platform standard for heterogeneous computing, supporting CPUs, GPUs, and other accelerators. It is maintained by the Khronos Group and offers portability but typically lower performance than vendor‑specific APIs like CUDA.
3.2.3 HIP
HIP (Heterogeneous‑Compute Interface for Portability) is AMD’s programming model designed to be compatible with CUDA syntax. It allows developers to write code that can be compiled for both AMD and NVIDIA GPUs, facilitating portability between platforms.
4 Applications
4.1 Gaming and Graphics
4.1.1 Real‑Time Rendering
GPUs are essential for real‑time rendering in video games, where they must generate 60 or more frames per second with complex geometry, lighting, and effects. Modern GPUs employ techniques like deferred shading, tessellation, and variable‑rate shading to maintain high performance.
4.1.2 Ray Tracing
Ray tracing simulates the physical behavior of light to produce photorealistic reflections, shadows, and global illumination. Dedicated hardware (e.g., NVIDIA RT Cores, AMD Ray Accelerators) accelerates bounding volume hierarchy traversal and ray‑triangle intersections, enabling real‑time ray tracing in games.
4.2 Scientific Computing
4.2.1 Machine Learning and AI
GPUs are the dominant platform for training deep neural networks due to their massive parallelism. Frameworks like TensorFlow, PyTorch, and JAX leverage CUDA and cuDNN (NVIDIA’s deep neural network library) to accelerate matrix multiplications, convolutions, and other operations. Tensor cores, available in NVIDIA GPUs, further speed up mixed‑precision training.
4.2.2 Scientific Simulations
GPU acceleration is applied to simulations in physics, chemistry, biology, and engineering. Examples include molecular dynamics (e.g., GROMACS, AMBER), computational fluid dynamics, and finite‑element analysis. The ability to run millions of threads simultaneously allows researchers to solve problems that would be impractical on CPUs alone.
4.3 Other Uses
4.3.1 Cryptocurrency Mining
GPUs were widely used for mining cryptocurrencies such as Bitcoin (early days), Ethereum, and Monero, which rely on proof‑of‑work algorithms that benefit from parallel hashing. The demand for GPUs during crypto booms caused shortages and price inflation in the consumer market, though many modern algorithms are now dominated by ASICs.
4.3.2 Video Encoding and Decoding
Most modern GPUs include dedicated hardware encoders (e.g., NVIDIA NVENC, AMD VCE, Intel Quick Sync Video) and decoders for video codecs like H.264, H.265/HEVC, and AV1. These units handle video compression and decompression with high efficiency, reducing CPU load during streaming, video editing, and playback.
5 Performance and Benchmarking
5.1 Key Metrics
5.1.1 FLOPS
FLOPS (floating‑point operations per second) measure the raw arithmetic throughput of a GPU. Single‑precision (FP32) and double‑precision (FP64) FLOPs are often quoted. Tensor cores also support lower‑precision formats (FP16, BF16, INT8) for AI workloads. Higher FLOPS generally indicate greater compute capability, but real‑world performance depends on workload characteristics.
5.1.2 Memory Bandwidth
Memory bandwidth, measured in GB/s, indicates the rate at which data can be read from or written to VRAM. It is critical for workloads that are memory‑bound, such as high‑resolution texturing, frame buffer operations, and large dataset processing. Bandwidth is determined by memory type (e.g., GDDR6, HBM2e) and bus width.
5.1.3 Clock Speed
GPU clock speeds (in MHz or GHz) represent the frequency at which the core operates. Boost clocks, which automatically increase under thermal and power limits, are commonly advertised. While clock speed affects performance, it is less indicative of overall capability than FLOPS and bandwidth due to architectural differences.
5.2 Benchmarking Tools
5.2.1 Synthetic Benchmarks
Synthetic benchmarks (e.g., 3DMark, Unigine Heaven, PassMark) stress specific GPU components under controlled conditions. They provide repeatable scores for comparing performance across different hardware, though they may not perfectly reflect real‑world application behavior.
5.2.2 Real‑World Benchmarks
Real‑world benchmarks measure performance in actual applications and games, such as frame rates in AAA titles, training times for neural networks, or render times in 3D software like Blender. These benchmarks are often published by hardware review sites and are considered more relevant for typical use cases.
6 Historical Development
6.1 Early Fixed‑Function GPUs
Before the late 1990s, GPUs were fixed‑function processors that performed a predefined set of operations (e.g., rasterization, texture mapping, blending) with no programmability. Examples include the 3dfx Voodoo series and NVIDIA RIVA TNT. These chips accelerated 3D graphics by handling specific pipeline stages in hardware.
6.2 Introduction of Programmability
The transition to programmable shaders began with NVIDIA’s GeForce 3 (2001), which introduced vertex shaders. Later, ATI’s Radeon 9700 (2002) added pixel shaders. Programmable shaders allowed developers to write custom code for vertex and pixel processing, enabling effects like per‑pixel lighting, bump mapping, and shadows.
6.2.1 Shader Model Evolution
Shader models, defined by DirectX, evolved from Shader Model 1.0 (simple vertex and pixel shaders) to Shader Model 3.0 (longer programs, dynamic branching) and Shader Model 4.0/5.0 (unified architecture, compute shaders). Each iteration increased instruction length, introduced new instruction sets, and expanded hardware capabilities.
6.3 Modern Era and Unified Architecture
The introduction of unified shader architecture (NVIDIA GeForce 8 series, 2006; AMD Radeon HD 2000 series, 2007) merged vertex and pixel shaders into a single pool of programmable ALUs. This allowed dynamic allocation of processing power to any stage of the pipeline, improving efficiency. The same architecture enabled general‑purpose computing via CUDA and later OpenCL.
6.3.1 Multi‑GPU Configurations
Multi‑GPU setups, such as NVIDIA SLI and AMD CrossFire, allowed multiple GPUs to work together on a single rendering task, increasing performance. However, scaling was imperfect and required driver support; by the late 2010s, these technologies were largely abandoned due to diminishing returns and the rise of single‑GPU solutions with sufficient power.
6.3.2 Tensor and RT Cores
NVIDIA introduced dedicated tensor cores in the Volta architecture (2017) for AI acceleration, and RT cores in the Turing architecture (2018) for real‑time ray tracing. These specialized units significantly improved performance in deep learning and ray‑tracing workloads, differentiating NVIDIA’s offerings. AMD followed with similar hardware (Ray Accelerators, AI accelerators) in later architectures.
7 Major Manufacturers
7.1 NVIDIA
NVIDIA Corporation, founded in 1993, is the dominant player in the GPU market. Its GeForce line targets gaming, while Quadro/RTX A-series serve professional markets, and Tesla (now renamed to some models) and HGX solutions are used for data‑center AI and HPC. NVIDIA’s CUDA ecosystem and software support give it a strong competitive advantage.
7.2 AMD (ATI)
Advanced Micro Devices (AMD) acquired ATI Technologies in 2006, inheriting the Radeon brand. AMD’s Radeon GPUs compete primarily with NVIDIA in gaming, and its Radeon Pro line targets workstations. AMD also designs semi‑custom GPUs for game consoles (PlayStation, Xbox) and integrated graphics in its Ryzen APUs. AMD emphasizes open‑source drivers and the FidelityFX suite.
7.3 Intel
Intel entered the discrete GPU market with its Intel Arc series in 2022, after decades of integrated graphics in its CPUs. Intel’s integrated GPUs (UHD Graphics, Iris Xe) are widely used in laptops, and the Arc series (Alchemist, Battlemage) targets mid‑range gaming and content creation.
7.3.1 Intel Arc Series
The Intel Arc series is built on the Xe HPG architecture, supporting hardware ray tracing, XeSS (Xe Super Sampling) upscaling, and AV1 encoding. Early adoption has been modest, but Intel continues to improve drivers and software support, aiming to become a third major competitor.
8 Future Trends
8.1 Advanced Packaging and Chiplets
Future GPUs are moving toward chiplet designs, where multiple smaller dies are interconnected using advanced packaging (e.g., EMIB, CoWoS, InFO). This approach improves yields, allows mixing of different process nodes, and enables scalable performance. AMD’s CDNA architectures and NVIDIA’s recent data‑center designs already employ chiplets.
8.2 On‑Chip AI Acceleration
AI acceleration is becoming a standard feature in GPUs, with dedicated tensor‑like units for low‑precision matrix operations. Future GPUs will likely integrate more flexible AI cores for tasks like real‑time neural rendering, denoising, and upscaling, as well as on‑device inference for local AI applications.
8.3 Growing Role in Edge Computing and Cloud Gaming
GPUs are increasingly deployed at the edge (e.g., automotive, IoT devices) for real‑time AI inference and video processing. In cloud gaming, remote GPU clusters stream rendered frames to thin clients, enabling high‑quality gaming on low‑end hardware. Technologies like NVIDIA GeForce NOW and AMD Cloud Gaming rely on scalable GPU infrastructure.