What is a CUDA core?

To understand CUDA cores, we first need to understand Compute Unified Device Architecture as a platform. Developed by Nvidia nearly 20 years ago, it’s a parallel computing platform for purpose-built APIs (Application Programming Interfaces) that lets developers access compilers and tools to run hardware-accelerated programs.

Supported programming languages for CUDA include C, C++, Fortran, Python, and Julia, with supported APIs including not only Direct3D and OpenGL, but specific frameworks such as OpenMP, OpenACC, and OpenCL. CUDA provides both low-level and higher-level APIs on its platform, with an ever-expanding list of libraries for generalized computing, which were previously only thought to be achieved through your computer’s processor.

A CUDA core is a SIMD (Single Instruction, Multiple Data) processing unit found inside your Nvidia graphics card that handles parallel computing tasks; with more CUDA cores, comes the ability to do more with your graphics card. The number of CUDA cores in today’s GPUs has steadily increased over the last 10 years, with top-end performers such as the RTX 5090 featuring 21,760 of them and the RTX 4090 using 16,384.

These two enthusiast-class graphics cards may be (primarily) marketed on their 4K and 8K gaming performance, but they’re also aimed at tasks such as data science, video processing, encoding, rendering, and AI model training.

Nvidia first created CUDA in 2006, with the first commercially available graphics cards to utilize the technology being the eighth generation of the original GeForce lineup, with the 8800 GTX later (featuring 128 CUDA cores).

Using CUDA, and its specifically developed API built on the platform, this GPU was significantly faster at general-purpose computing outside of just traditional graphics rendering, which were the sole point of video cards back in the day.

Every Nvidia graphics card released afterwards, including the GeForce 500 series, GeForce 600 series, GeForce 700 series, and GeForce 900 series, was built to support CUDA.

Around this time, we saw graphics cards begin to be fully marketed around their CUDA-capable prowess for advanced computing, such as with the Nvidia GTX Titan in 2013, which featured 2,688 CUDA cores and 6GB GDDR5 memory at a time when its contemporaries (like the GTX 770 and GTX 780) lagged significantly.

Fast-forwarding to today, thousands of applications have been developed with CUDA, and all graphics cards from Nvidia natively support the platform, whether they’re gaming GPUs (like the RTX 5070 and RTX 5080) or high-end Quadro ones made expressly for developers and data servers.

The CUDA Toolkit has been steadily upgraded since its launch in 2007, where it’s currently in its 12th iteration, which is primarily made for the company’s H100 and A100 GPUs, with new APIs and tools specific to data center platforms.

CUDA cores work similarly to how CPU cores work on a desktop or laptop processor; they’re built to process large amounts of data simultaneously with a technique called SIMT (Single Instruction, Multiple Threads). In essence, this means a large number of cores all working on an identical process at the same time.

Whereas some of the best processors on the market (like the AMD Ryzen 9 9950X3D) may feature 16 processing cores, the average GPU now features around 3,000 processing cores, making hardware-based (GPU-accelerated) tasks, such as video editing, 3D rendering, gaming, and simulation, easier and faster to do.

Whereas a CPU core has lower latency and is good for serial processing, a CUDA core has higher throughput and breaks down the processes into smaller tasks through parallel processing.

As its name suggests, many thousands of CUDA cores built into your GPU execute the same process, synchronizing the sub-tasks independently. CUDA cores are, therefore, highly specialized for specific tasks compared to a CPU’s more generalized approach.

Considering that CUDA cores are parallel processing units that excel at large and intensive operations, having more of them can make your gaming experience smoother and faster.

They handle advanced calculations such as lightning, shading, physics, rasterization, pixel calculating, anti-aliasing, frame rate optimization, texture mapping, and more. With parallel computing, these intensive tasks can be broken down into smaller jobs that the CUDA cores work through all at once.

For more advanced computing processing, CUDA cores can do things such as high-level data processing, scientific simulations, and mathematical operations, because of how a CUDA core executes a floating point and integer operation concurrently.

CUDA as a platform has been praised for its C/C++ interface, ease of use, large ecosystem, libraries, and existing programming models, and there are nearly 20 years of hardware developed to fall back on it. Everything from image processing, deep learning, and other forms of computational science can be achieved with the platform, after all.

CUDA is an Nvidia-developed platform, and CUDA cores are the company’s term for its GPU cores. AMD utilizes completely different Stream Processors for its GPU cores, which do not equate to one another.

To boil things down to the most basic comparison, both CUDA cores and Stream Processors are essentially just shaders (or Unified Shader Units), which are capable of parallel computing tasks, such as shading, etc.