{"schemaVersion":1,"id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute","title":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication","description":"Warp divergence, shared memory bank conflicts, occupancy tuning, and an optimized matrix multiplication kernel with benchmarks across NVIDIA, AMD, and Vulkan Compute.","language":"en","datePublished":"2026-03-06","dateModified":null,"contentFormat":"text/markdown","content":"GPU Computing\n\n# GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication\n\n2026-03-0612 min read\n\n[Lucio Durán](<https://lucioduran.com/>)\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/gpu-programming-cuda-vs-rocm-vulkan-compute>)\n\n## The Execution Model: Warps, Wavefronts, and Subgroups\n\nThe following describes what occurs when code executes on the GPU.\n\n### NVIDIA (CUDA): Warps of 32\n\nThe fundamental execution unit on NVIDIA ([CUDA](<https://developer.nvidia.com/cuda-toolkit>)) is the **warp**, 32 threads that execute the same instruction in lockstep (SIMT: Single Instruction, Multiple Threads). A Streaming Multiprocessor (SM) can schedule multiple warps, but within a single warp, all 32 threads are at the same program counter.\n\n```cuda\n// This looks innocent but murders performance\n__global__ void bad_branch(float* data, int n) {\n int tid = threadIdx.x + blockIdx.x * blockDim.x;\n if (tid < n) {\n if (data[tid] > 0.0f) {\n data[tid] = sqrtf(data[tid]); // Path A\n } else {\n data[tid] = -sqrtf(-data[tid]); // Path B\n }\n }\n}\n// If threads 0-15 take Path A and threads 16-31 take Path B,\n// the warp executes BOTH paths sequentially. 50% utilization.\n```\n\n### AMD (ROCm/HIP): Wavefronts of 64\n\nAMD's equivalent is the **wavefront**, 64 threads on CDNA/RDNA architectures. This is not just a naming difference; it fundamentally changes how you think about occupancy and warp-level primitives.\n\n```cpp\n// HIP code, looks identical to CUDA but wavefront = 64\n__global__ void hip_reduce(float* data, float* result, int n) {\n __shared__ float sdata[256];\n int tid = threadIdx.x;\n\n sdata[tid] = (tid < n) ? data[tid] : 0.0f;\n __syncthreads();\n\n // DANGER: this reduction assumes warp size of 32\n // On AMD, the last 'unrolled' iterations need to cover 64 threads\n for (int s = blockDim.x / 2; s > 32; s >>= 1) {\n if (tid < s) sdata[tid] += sdata[tid + s];\n __syncthreads();\n }\n\n // This warp-level unrolling is WRONG on AMD\n if (tid < 32) { // Should be < 64 on AMD!\n volatile float* vsmem = sdata;\n vsmem[tid] += vsmem[tid + 32];\n vsmem[tid] += vsmem[tid + 16];\n vsmem[tid] += vsmem[tid + 8];\n vsmem[tid] += vsmem[tid + 4];\n vsmem[tid] += vsmem[tid + 2];\n vsmem[tid] += vsmem[tid + 1];\n }\n}\n```\n\nThis exact bug surfaces when porting CUDA codebases to ROCm for MI250X clusters. ` hipify-perl ` translates the syntax perfectly but does not flag the warp-size assumption. The results can be wrong by ~3%, close enough to pass unit tests with loose tolerances but completely wrong for physics simulations.\n\n### Vulkan Compute: Subgroups\n\nVulkan doesn't specify a subgroup size. It could be 4, 8, 16, 32, 64, or even 128 depending on the hardware and driver. You query it at runtime:\n\n```glsl\n#version 450\n#extension GL_KHR_shader_subgroup_basic : enable\n#extension GL_KHR_shader_subgroup_arithmetic : enable\n\nlayout(local_size_x = 256) in;\nlayout(set = 0, binding = 0) buffer Data { float data[]; };\nlayout(set = 0, binding = 1) buffer Result { float result; };\n\nvoid main() {\n uint tid = gl_LocalInvocationID.x;\n\n // subgroupSize is a built-in, could be 32, 64, etc.\n float val = data[tid];\n float subgroup_sum = subgroupAdd(val);\n\n // Only first thread in each subgroup writes\n if (subgroupElect()) {\n // Atomic add the subgroup's partial sum\n atomicAdd(result, subgroup_sum);\n }\n}\n```\n\nThe portability is nice but the performance tuning is harder. You can't hard-code assumptions about subgroup size, which means your optimization strategies need to be parametric.\n\n## Shared Memory: The Make-or-Break Resource\n\nShared memory (CUDA/HIP) or Shared Storage (Vulkan) is the fast on-chip SRAM that makes GPU computing practical. On modern NVIDIA GPUs, it's organized into 32 banks, each 4 bytes wide.\n\n### Bank Conflicts: The Silent Killer\n\n```cuda\n// Matrix stored in shared memory: 32 x 32 floats\n__shared__ float tile[32][32];\n\n// Thread (tx, ty) reads column-wise:\n// Thread 0 reads tile[0][0] -> bank 0\n// Thread 1 reads tile[1][0] -> bank 0 (!!!)\n// Thread 2 reads tile[2][0] -> bank 0 (!!!)\n// ... all 32 threads hit bank 0 -> 32-way conflict!\n\n// Fix: pad each row by 1 element\n__shared__ float tile[32][33]; // <-- the magic number\n\n// Now:\n// Thread 0 reads tile[0][0] -> bank 0\n// Thread 1 reads tile[1][0] -> bank 1 (offset by 33 % 32 = 1)\n// Thread 2 reads tile[2][0] -> bank 2\n// ... all 32 threads hit different banks -> no conflict!\n```\n\nThat ` [32][33] ` instead of ` [32][32] ` is the kind of thing that turns a 40% peak kernel into an 80% peak kernel. One extra element per row. This single change can give 2x speedups on memory-bound kernels.\n\n## Matrix Multiplication: Optimized SGEMM\n\nThe naive triple loop is insufficient for production use. The following tiled SGEMM kernel has been iteratively optimized. This version gets ~75% of cuBLAS performance on an A100, which is decent for hand-written code.\n\n```cuda\n// Tiled SGEMM: C = A * B\n// A is M x K, B is K x N, C is M x N\n// Block tile: BM x BN, Thread tile: TM x TN\n\n#define BM 128\n#define BN 128\n#define BK 8\n#define TM 8\n#define TN 8\n\n__global__ void sgemm_optimized(\n const float* __restrict__ A,\n const float* __restrict__ B,\n float* __restrict__ C,\n int M, int N, int K\n) {\n // Block and thread indexing\n const int bx = blockIdx.x;\n const int by = blockIdx.y;\n const int tx = threadIdx.x; // 0..15\n const int ty = threadIdx.y; // 0..15\n\n // Shared memory tiles with padding to avoid bank conflicts\n __shared__ float As[BK][BM + 1]; // Transposed for coalescing\n __shared__ float Bs[BK][BN + 1];\n\n // Each thread computes a TM x TN sub-tile of C\n float accum[TM][TN] = {0.0f};\n\n // Register cache for A and B fragments\n float a_reg[TM];\n float b_reg[TN];\n\n // Base pointers\n const int row_base = by * BM;\n const int col_base = bx * BN;\n\n // Loop over K dimension in BK-sized tiles\n for (int k = 0; k < K; k += BK) {\n // Collaborative loading of A tile into shared memory\n // Each thread loads multiple elements\n #pragma unroll\n for (int i = 0; i < BM; i += blockDim.y) {\n int row = row_base + ty + i;\n int col = k + tx;\n if (row < M && col < K) {\n // Store transposed for bank-conflict-free access later\n As[tx][ty + i] = A[row * K + col];\n } else {\n As[tx][ty + i] = 0.0f;\n }\n }\n\n // Collaborative loading of B tile\n #pragma unroll\n for (int i = 0; i < BN; i += blockDim.y) {\n int row = k + ty;\n int col = col_base + tx + i;\n if (row < K && col < N) {\n Bs[ty][tx + i] = B[row * N + col];\n } else {\n Bs[ty][tx + i] = 0.0f;\n }\n }\n\n __syncthreads();\n\n // Compute TM x TN partial results\n #pragma unroll\n for (int kk = 0; kk < BK; kk++) {\n // Load A fragment into registers\n #pragma unroll\n for (int m = 0; m < TM; m++) {\n a_reg[m] = As[kk][ty * TM + m];\n }\n // Load B fragment into registers\n #pragma unroll\n for (int n = 0; n < TN; n++) {\n b_reg[n] = Bs[kk][tx * TN + n];\n }\n // Outer product accumulation\n #pragma unroll\n for (int m = 0; m < TM; m++) {\n #pragma unroll\n for (int n = 0; n < TN; n++) {\n accum[m][n] += a_reg[m] * b_reg[n];\n }\n }\n }\n\n __syncthreads();\n }\n\n // Write results back to global memory\n #pragma unroll\n for (int m = 0; m < TM; m++) {\n #pragma unroll\n for (int n = 0; n < TN; n++) {\n int row = row_base + ty * TM + m;\n int col = col_base + tx * TN + n;\n if (row < M && col < N) {\n C[row * N + col] = accum[m][n];\n }\n }\n }\n}\n```\n\nKey optimization decisions in this kernel:\n\n1. **Transposed shared memory store for A**, When threads read a column of ` As `, they access consecutive rows which maps to different banks.\n2. **Register tiling (TM x TN)**, Each thread computes an 8x8 sub-tile, amortizing shared memory loads across 64 FMA operations per K step.\n3. **` __restrict__ ` pointers**, Tells the compiler A, B, C don't alias, enabling more aggressive optimization.\n4. **` #pragma unroll `**, Critical for the inner loops; without it the compiler may not fully unroll and you lose ILP.\n\n## The ROCm Port: Not Just Find-and-Replace\n\nWhen porting this to HIP for MI250X, several non-obvious issues arise:\n\n```cpp\n// HIP version, changes marked with comments\n#include <hip/hip_runtime.h>\n\n#define BM 128\n#define BN 128\n#define BK 8 // May need to be 16 on MI250X for better occupancy\n#define TM 8\n#define TN 8\n\n__global__ void sgemm_hip(\n const float* __restrict__ A,\n const float* __restrict__ B,\n float* __restrict__ C,\n int M, int N, int K\n) {\n // AMD CDNA uses 64-wide wavefronts\n // Occupancy calculation differs:\n // MI250X has 104 CUs, each can run up to 40 wavefronts\n // Shared memory per CU: 64 KB (vs 164 KB on A100 with opt-in)\n\n // Same kernel structure works but optimal tile sizes differ\n // because of wavefront size and shared memory capacity\n\n // ... (same structure as CUDA version, but compiled with hipcc)\n}\n\n// Launch config also differs\nvoid launch_sgemm(float* A, float* B, float* C, int M, int N, int K) {\n dim3 block(BN / TN, BM / TM); // 16 x 16 = 256 threads\n dim3 grid((N + BN - 1) / BN, (M + BM - 1) / BM);\n\n // HIP equivalent of cudaFuncSetAttribute\n hipFuncSetAttribute(\n (const void*)sgemm_hip,\n hipFuncAttributeMaxDynamicSharedMemorySize,\n 65536 // Request 64KB if needed\n );\n\n hipLaunchKernelGGL(sgemm_hip, grid, block, 0, 0,\n A, B, C, M, N, K);\n}\n```\n\nThe big gotcha: AMD's shared memory is 64 KB per CU vs NVIDIA's configurable 164 KB on A100. If your tiling strategy needs more than 64 KB of shared memory, you need smaller tiles on AMD, which changes your entire optimization strategy.\n\n## Vulkan Compute: The Portable Alternative\n\nHere's the same matrix multiply concept in Vulkan Compute GLSL:\n\n```glsl\n#version 450\n#extension GL_KHR_shader_subgroup_basic : enable\n\nlayout(local_size_x = 16, local_size_y = 16) in;\n\nlayout(set = 0, binding = 0) readonly buffer MatA { float A[]; };\nlayout(set = 0, binding = 1) readonly buffer MatB { float B[]; };\nlayout(set = 0, binding = 2) writeonly buffer MatC { float C[]; };\n\nlayout(push_constant) uniform Params {\n uint M, N, K;\n};\n\n// Tile size in shared memory\n#define TILE_SIZE 16\n\nshared float tileA[TILE_SIZE][TILE_SIZE + 1]; // +1 padding\nshared float tileB[TILE_SIZE][TILE_SIZE + 1];\n\nvoid main() {\n uint row = gl_GlobalInvocationID.y;\n uint col = gl_GlobalInvocationID.x;\n uint localRow = gl_LocalInvocationID.y;\n uint localCol = gl_LocalInvocationID.x;\n\n float sum = 0.0;\n\n for (uint t = 0; t < (K + TILE_SIZE - 1) / TILE_SIZE; t++) {\n uint tiledCol = t * TILE_SIZE + localCol;\n uint tiledRow = t * TILE_SIZE + localRow;\n\n tileA[localRow][localCol] = (row < M && tiledCol < K)\n ? A[row * K + tiledCol] : 0.0;\n tileB[localRow][localCol] = (tiledRow < K && col < N)\n ? B[tiledRow * N + col] : 0.0;\n\n barrier();\n\n for (uint k = 0; k < TILE_SIZE; k++) {\n sum += tileA[localRow][k] * tileB[k][localCol];\n }\n\n barrier();\n }\n\n if (row < M && col < N) {\n C[row * N + col] = sum;\n }\n}\n```\n\nThis is simpler but also slower, you can't do register tiling as aggressively in GLSL, and the compiler's optimizer varies wildly between GPU vendors.\n\n## Benchmarks: Real Numbers on Real Hardware\n\nThe following benchmarks show 4096x4096 SGEMM on three setups:\n\n| Implementation | Hardware | TFLOPS | % of Peak |\n| --- | --- | --- | --- |\n| cuBLAS | A100 80GB | 17.8 | 91% |\n| Custom CUDA kernel | A100 80GB | 14.7 | 75% |\n| rocBLAS | MI250X | 15.2 | 88% |\n| Custom HIP kernel | MI250X | 11.9 | 69% |\n| Vulkan Compute (GLSL) | RTX 4090 | 9.3 | 56% |\n| Vulkan Compute (GLSL) | RX 7900 XTX | 7.1 | 52% |\n\nThe vendor libraries (cuBLAS, rocBLAS) are doing assembly-level tricks that are impractical to replicate in high-level code, things like software pipelining of shared memory loads and explicit register allocation. But getting 75% of peak in portable C++ is respectable and often sufficient for custom kernels where cuBLAS doesn't have an exact routine.\n\n## Profiling: Where the Time Actually Goes\n\nThe single most valuable skill in GPU programming is reading a profiler output. For the CUDA kernel, Nsight Compute reports:\n\n```\nKernel: sgemm_optimized\nDuration: 2.31 ms\nTheoretical Occupancy: 75%\nAchieved Occupancy: 71.2%\nSM Throughput: 87.3%\nMemory Throughput: 62.1% <-- Not memory bound\nCompute Throughput: 87.3% <-- Compute bound\n\nL1/TEX Hit Rate: 94.7%\nL2 Hit Rate: 78.3%\nShared Memory Bank Conflicts: 0 <-- The padding worked\n\nWarp Stall Reasons:\n stall_mio_throttle: 23.1% <-- Shared memory bandwidth\n stall_math_pipe_throttle: 18.7% <-- Good! ALU is saturated\n stall_short_scoreboard: 15.2% <-- Waiting for MIO results\n stall_not_selected: 12.4% <-- Scheduler couldn't pick this warp\n```\n\nThat ` stall_mio_throttle ` at 23.1% indicates shared memory bandwidth is the next bottleneck, the kernel loads from shared memory faster than the hardware can serve. The fix would be to increase the register tile size (TM/TN) to do more computation per shared memory load, at the cost of higher register pressure and potentially lower occupancy.\n\nThis is the game. Every optimization creates a new bottleneck. The work is done when the profiler shows a limitation by something fundamental, ALU throughput, memory bandwidth, or occupancy limits from register pressure.\n\n## Decision Framework\n\nAn honest assessment of all three ecosystems after extensive use:\n\n**CUDA** has the best tooling by far. Nsight Compute, Nsight Systems, cuda-memcheck, the debugging and profiling story is unmatched. If you're on NVIDIA hardware, there's no reason to use anything else for compute.\n\n**ROCm** has improved dramatically but still feels 2 years behind CUDA in tooling. The ` rocprof ` profiler is usable but nowhere near Nsight Compute. The HIP language itself is 98% source-compatible with CUDA though, which makes porting straightforward. Worth it if you have AMD hardware or need to avoid NVIDIA lock-in.\n\n**Vulkan Compute** is the right choice when you need one codebase running on Intel, NVIDIA, AMD, and even mobile GPUs. The trade-off is less control over hardware specifics and a more verbose API. For ML inference engines that need to run everywhere, it's increasingly compelling.\n\nSelect based on project constraints, not vendor loyalty. Profile before optimizing.\n\n## Frequently asked questions\n\n### When should I use CUDA vs ROCm vs Vulkan Compute?\n\nCUDA if you're NVIDIA-only and want the best tooling and ecosystem. ROCm if you're targeting AMD GPUs (especially MI250/MI300 for ML workloads). Vulkan Compute if you need cross-vendor portability or you're doing GPU compute in a context that already uses Vulkan for graphics.\n\n### What is a warp divergence and why does it kill performance?\n\nA warp (32 threads on NVIDIA, 64 on AMD as a 'wavefront') executes in lockstep. When threads in the same warp take different branches in an if/else, the hardware serializes both paths, threads not on the active path are masked off but still consume cycles. This can halve your throughput or worse.\n\n### How do shared memory bank conflicts affect performance?\n\nShared memory is divided into 32 banks. When multiple threads in a warp access different addresses that map to the same bank simultaneously, the accesses are serialized. A full 32-way bank conflict means 32x slower access. Padding your shared memory arrays by 1 element per row often eliminates these conflicts.\n\n## Tips\n\n1. **Tip:** Profile with ` ncu --set full ` (Nsight Compute) before touching your kernel code. Teams often spend days 'optimizing' a kernel that was memory-bound, adding more arithmetic instructions that the scheduler just hid behind memory latency anyway. Know your bottleneck first.\n2. **Tip:** On AMD GPUs, the wavefront size is 64 (not 32 like CUDA warps). If you port CUDA code to ROCm with ` hipify-perl `, audit every ` __shfl_sync ` and warp-level primitive manually, the automatic translation gets the syntax right but the semantics wrong for algorithms that depend on warp size.\n3. **Tip:** For Vulkan Compute, always set ` VK_KHR_shader_non_semantic_info ` in your device extensions and compile GLSL with ` -g ` flag. Without it, RenderDoc shows your compute shader as an opaque blob and debugging becomes guesswork.\n\n- cuda\n- rocm\n- vulkan-compute\n- gpu-programming\n- matrix-multiplication\n- warp-scheduling\n- shared-memory\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#title"},{"level":2,"text":"The Execution Model: Warps, Wavefronts, and Subgroups","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#the-execution-model-warps-wavefronts-and-subgroups"},{"level":3,"text":"NVIDIA (CUDA): Warps of 32","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#nvidia-cuda-warps-of-32"},{"level":3,"text":"AMD (ROCm/HIP): Wavefronts of 64","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#amd-rocmhip-wavefronts-of-64"},{"level":3,"text":"Vulkan Compute: Subgroups","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#vulkan-compute-subgroups"},{"level":2,"text":"Shared Memory: The Make-or-Break Resource","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#shared-memory-the-make-or-break-resource"},{"level":3,"text":"Bank Conflicts: The Silent Killer","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#bank-conflicts-the-silent-killer"},{"level":2,"text":"Matrix Multiplication: Optimized SGEMM","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#matrix-multiplication-optimized-sgemm"},{"level":2,"text":"The ROCm Port: Not Just Find-and-Replace","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#the-rocm-port-not-just-find-and-replace"},{"level":2,"text":"Vulkan Compute: The Portable Alternative","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#vulkan-compute-the-portable-alternative"},{"level":2,"text":"Benchmarks: Real Numbers on Real Hardware","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#benchmarks-real-numbers-on-real-hardware"},{"level":2,"text":"Profiling: Where the Time Actually Goes","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#profiling-where-the-time-actually-goes"},{"level":2,"text":"Decision Framework","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#decision-framework"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#questions"},{"level":3,"text":"When should I use CUDA vs ROCm vs Vulkan Compute?","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#question-1"},{"level":3,"text":"What is a warp divergence and why does it kill performance?","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#question-2"},{"level":3,"text":"How do shared memory bank conflicts affect performance?","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#article-tips"}],"sections":[{"heading":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication","level":1,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute","content":"GPU Computing"},{"heading":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication","level":1,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#title","content":"2026-03-0612 min read\n\n[Lucio Durán](<https://lucioduran.com/>)\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/gpu-programming-cuda-vs-rocm-vulkan-compute>)"},{"heading":"The Execution Model: Warps, Wavefronts, and Subgroups","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#the-execution-model-warps-wavefronts-and-subgroups","content":"The following describes what occurs when code executes on the GPU."},{"heading":"NVIDIA (CUDA): Warps of 32","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#nvidia-cuda-warps-of-32","content":"The fundamental execution unit on NVIDIA ([CUDA](<https://developer.nvidia.com/cuda-toolkit>)) is the **warp**, 32 threads that execute the same instruction in lockstep (SIMT: Single Instruction, Multiple Threads). A Streaming Multiprocessor (SM) can schedule multiple warps, but within a single warp, all 32 threads are at the same program counter.\n\n```cuda\n// This looks innocent but murders performance\n__global__ void bad_branch(float* data, int n) {\n int tid = threadIdx.x + blockIdx.x * blockDim.x;\n if (tid < n) {\n if (data[tid] > 0.0f) {\n data[tid] = sqrtf(data[tid]); // Path A\n } else {\n data[tid] = -sqrtf(-data[tid]); // Path B\n }\n }\n}\n// If threads 0-15 take Path A and threads 16-31 take Path B,\n// the warp executes BOTH paths sequentially. 50% utilization.\n```"},{"heading":"AMD (ROCm/HIP): Wavefronts of 64","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#amd-rocmhip-wavefronts-of-64","content":"AMD's equivalent is the **wavefront**, 64 threads on CDNA/RDNA architectures. This is not just a naming difference; it fundamentally changes how you think about occupancy and warp-level primitives.\n\n```cpp\n// HIP code, looks identical to CUDA but wavefront = 64\n__global__ void hip_reduce(float* data, float* result, int n) {\n __shared__ float sdata[256];\n int tid = threadIdx.x;\n\n sdata[tid] = (tid < n) ? data[tid] : 0.0f;\n __syncthreads();\n\n // DANGER: this reduction assumes warp size of 32\n // On AMD, the last 'unrolled' iterations need to cover 64 threads\n for (int s = blockDim.x / 2; s > 32; s >>= 1) {\n if (tid < s) sdata[tid] += sdata[tid + s];\n __syncthreads();\n }\n\n // This warp-level unrolling is WRONG on AMD\n if (tid < 32) { // Should be < 64 on AMD!\n volatile float* vsmem = sdata;\n vsmem[tid] += vsmem[tid + 32];\n vsmem[tid] += vsmem[tid + 16];\n vsmem[tid] += vsmem[tid + 8];\n vsmem[tid] += vsmem[tid + 4];\n vsmem[tid] += vsmem[tid + 2];\n vsmem[tid] += vsmem[tid + 1];\n }\n}\n```\n\nThis exact bug surfaces when porting CUDA codebases to ROCm for MI250X clusters. ` hipify-perl ` translates the syntax perfectly but does not flag the warp-size assumption. The results can be wrong by ~3%, close enough to pass unit tests with loose tolerances but completely wrong for physics simulations."},{"heading":"Vulkan Compute: Subgroups","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#vulkan-compute-subgroups","content":"Vulkan doesn't specify a subgroup size. It could be 4, 8, 16, 32, 64, or even 128 depending on the hardware and driver. You query it at runtime:\n\n```glsl\n#version 450\n#extension GL_KHR_shader_subgroup_basic : enable\n#extension GL_KHR_shader_subgroup_arithmetic : enable\n\nlayout(local_size_x = 256) in;\nlayout(set = 0, binding = 0) buffer Data { float data[]; };\nlayout(set = 0, binding = 1) buffer Result { float result; };\n\nvoid main() {\n uint tid = gl_LocalInvocationID.x;\n\n // subgroupSize is a built-in, could be 32, 64, etc.\n float val = data[tid];\n float subgroup_sum = subgroupAdd(val);\n\n // Only first thread in each subgroup writes\n if (subgroupElect()) {\n // Atomic add the subgroup's partial sum\n atomicAdd(result, subgroup_sum);\n }\n}\n```\n\nThe portability is nice but the performance tuning is harder. You can't hard-code assumptions about subgroup size, which means your optimization strategies need to be parametric."},{"heading":"Shared Memory: The Make-or-Break Resource","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#shared-memory-the-make-or-break-resource","content":"Shared memory (CUDA/HIP) or Shared Storage (Vulkan) is the fast on-chip SRAM that makes GPU computing practical. On modern NVIDIA GPUs, it's organized into 32 banks, each 4 bytes wide."},{"heading":"Bank Conflicts: The Silent Killer","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#bank-conflicts-the-silent-killer","content":"```cuda\n// Matrix stored in shared memory: 32 x 32 floats\n__shared__ float tile[32][32];\n\n// Thread (tx, ty) reads column-wise:\n// Thread 0 reads tile[0][0] -> bank 0\n// Thread 1 reads tile[1][0] -> bank 0 (!!!)\n// Thread 2 reads tile[2][0] -> bank 0 (!!!)\n// ... all 32 threads hit bank 0 -> 32-way conflict!\n\n// Fix: pad each row by 1 element\n__shared__ float tile[32][33]; // <-- the magic number\n\n// Now:\n// Thread 0 reads tile[0][0] -> bank 0\n// Thread 1 reads tile[1][0] -> bank 1 (offset by 33 % 32 = 1)\n// Thread 2 reads tile[2][0] -> bank 2\n// ... all 32 threads hit different banks -> no conflict!\n```\n\nThat ` [32][33] ` instead of ` [32][32] ` is the kind of thing that turns a 40% peak kernel into an 80% peak kernel. One extra element per row. This single change can give 2x speedups on memory-bound kernels."},{"heading":"Matrix Multiplication: Optimized SGEMM","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#matrix-multiplication-optimized-sgemm","content":"The naive triple loop is insufficient for production use. The following tiled SGEMM kernel has been iteratively optimized. This version gets ~75% of cuBLAS performance on an A100, which is decent for hand-written code.\n\n```cuda\n// Tiled SGEMM: C = A * B\n// A is M x K, B is K x N, C is M x N\n// Block tile: BM x BN, Thread tile: TM x TN\n\n#define BM 128\n#define BN 128\n#define BK 8\n#define TM 8\n#define TN 8\n\n__global__ void sgemm_optimized(\n const float* __restrict__ A,\n const float* __restrict__ B,\n float* __restrict__ C,\n int M, int N, int K\n) {\n // Block and thread indexing\n const int bx = blockIdx.x;\n const int by = blockIdx.y;\n const int tx = threadIdx.x; // 0..15\n const int ty = threadIdx.y; // 0..15\n\n // Shared memory tiles with padding to avoid bank conflicts\n __shared__ float As[BK][BM + 1]; // Transposed for coalescing\n __shared__ float Bs[BK][BN + 1];\n\n // Each thread computes a TM x TN sub-tile of C\n float accum[TM][TN] = {0.0f};\n\n // Register cache for A and B fragments\n float a_reg[TM];\n float b_reg[TN];\n\n // Base pointers\n const int row_base = by * BM;\n const int col_base = bx * BN;\n\n // Loop over K dimension in BK-sized tiles\n for (int k = 0; k < K; k += BK) {\n // Collaborative loading of A tile into shared memory\n // Each thread loads multiple elements\n #pragma unroll\n for (int i = 0; i < BM; i += blockDim.y) {\n int row = row_base + ty + i;\n int col = k + tx;\n if (row < M && col < K) {\n // Store transposed for bank-conflict-free access later\n As[tx][ty + i] = A[row * K + col];\n } else {\n As[tx][ty + i] = 0.0f;\n }\n }\n\n // Collaborative loading of B tile\n #pragma unroll\n for (int i = 0; i < BN; i += blockDim.y) {\n int row = k + ty;\n int col = col_base + tx + i;\n if (row < K && col < N) {\n Bs[ty][tx + i] = B[row * N + col];\n } else {\n Bs[ty][tx + i] = 0.0f;\n }\n }\n\n __syncthreads();\n\n // Compute TM x TN partial results\n #pragma unroll\n for (int kk = 0; kk < BK; kk++) {\n // Load A fragment into registers\n #pragma unroll\n for (int m = 0; m < TM; m++) {\n a_reg[m] = As[kk][ty * TM + m];\n }\n // Load B fragment into registers\n #pragma unroll\n for (int n = 0; n < TN; n++) {\n b_reg[n] = Bs[kk][tx * TN + n];\n }\n // Outer product accumulation\n #pragma unroll\n for (int m = 0; m < TM; m++) {\n #pragma unroll\n for (int n = 0; n < TN; n++) {\n accum[m][n] += a_reg[m] * b_reg[n];\n }\n }\n }\n\n __syncthreads();\n }\n\n // Write results back to global memory\n #pragma unroll\n for (int m = 0; m < TM; m++) {\n #pragma unroll\n for (int n = 0; n < TN; n++) {\n int row = row_base + ty * TM + m;\n int col = col_base + tx * TN + n;\n if (row < M && col < N) {\n C[row * N + col] = accum[m][n];\n }\n }\n }\n}\n```\n\nKey optimization decisions in this kernel:\n\n1. **Transposed shared memory store for A**, When threads read a column of ` As `, they access consecutive rows which maps to different banks.\n2. **Register tiling (TM x TN)**, Each thread computes an 8x8 sub-tile, amortizing shared memory loads across 64 FMA operations per K step.\n3. **` __restrict__ ` pointers**, Tells the compiler A, B, C don't alias, enabling more aggressive optimization.\n4. **` #pragma unroll `**, Critical for the inner loops; without it the compiler may not fully unroll and you lose ILP."},{"heading":"The ROCm Port: Not Just Find-and-Replace","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#the-rocm-port-not-just-find-and-replace","content":"When porting this to HIP for MI250X, several non-obvious issues arise:\n\n```cpp\n// HIP version, changes marked with comments\n#include <hip/hip_runtime.h>\n\n#define BM 128\n#define BN 128\n#define BK 8 // May need to be 16 on MI250X for better occupancy\n#define TM 8\n#define TN 8\n\n__global__ void sgemm_hip(\n const float* __restrict__ A,\n const float* __restrict__ B,\n float* __restrict__ C,\n int M, int N, int K\n) {\n // AMD CDNA uses 64-wide wavefronts\n // Occupancy calculation differs:\n // MI250X has 104 CUs, each can run up to 40 wavefronts\n // Shared memory per CU: 64 KB (vs 164 KB on A100 with opt-in)\n\n // Same kernel structure works but optimal tile sizes differ\n // because of wavefront size and shared memory capacity\n\n // ... (same structure as CUDA version, but compiled with hipcc)\n}\n\n// Launch config also differs\nvoid launch_sgemm(float* A, float* B, float* C, int M, int N, int K) {\n dim3 block(BN / TN, BM / TM); // 16 x 16 = 256 threads\n dim3 grid((N + BN - 1) / BN, (M + BM - 1) / BM);\n\n // HIP equivalent of cudaFuncSetAttribute\n hipFuncSetAttribute(\n (const void*)sgemm_hip,\n hipFuncAttributeMaxDynamicSharedMemorySize,\n 65536 // Request 64KB if needed\n );\n\n hipLaunchKernelGGL(sgemm_hip, grid, block, 0, 0,\n A, B, C, M, N, K);\n}\n```\n\nThe big gotcha: AMD's shared memory is 64 KB per CU vs NVIDIA's configurable 164 KB on A100. If your tiling strategy needs more than 64 KB of shared memory, you need smaller tiles on AMD, which changes your entire optimization strategy."},{"heading":"Vulkan Compute: The Portable Alternative","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#vulkan-compute-the-portable-alternative","content":"Here's the same matrix multiply concept in Vulkan Compute GLSL:\n\n```glsl\n#version 450\n#extension GL_KHR_shader_subgroup_basic : enable\n\nlayout(local_size_x = 16, local_size_y = 16) in;\n\nlayout(set = 0, binding = 0) readonly buffer MatA { float A[]; };\nlayout(set = 0, binding = 1) readonly buffer MatB { float B[]; };\nlayout(set = 0, binding = 2) writeonly buffer MatC { float C[]; };\n\nlayout(push_constant) uniform Params {\n uint M, N, K;\n};\n\n// Tile size in shared memory\n#define TILE_SIZE 16\n\nshared float tileA[TILE_SIZE][TILE_SIZE + 1]; // +1 padding\nshared float tileB[TILE_SIZE][TILE_SIZE + 1];\n\nvoid main() {\n uint row = gl_GlobalInvocationID.y;\n uint col = gl_GlobalInvocationID.x;\n uint localRow = gl_LocalInvocationID.y;\n uint localCol = gl_LocalInvocationID.x;\n\n float sum = 0.0;\n\n for (uint t = 0; t < (K + TILE_SIZE - 1) / TILE_SIZE; t++) {\n uint tiledCol = t * TILE_SIZE + localCol;\n uint tiledRow = t * TILE_SIZE + localRow;\n\n tileA[localRow][localCol] = (row < M && tiledCol < K)\n ? A[row * K + tiledCol] : 0.0;\n tileB[localRow][localCol] = (tiledRow < K && col < N)\n ? B[tiledRow * N + col] : 0.0;\n\n barrier();\n\n for (uint k = 0; k < TILE_SIZE; k++) {\n sum += tileA[localRow][k] * tileB[k][localCol];\n }\n\n barrier();\n }\n\n if (row < M && col < N) {\n C[row * N + col] = sum;\n }\n}\n```\n\nThis is simpler but also slower, you can't do register tiling as aggressively in GLSL, and the compiler's optimizer varies wildly between GPU vendors."},{"heading":"Benchmarks: Real Numbers on Real Hardware","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#benchmarks-real-numbers-on-real-hardware","content":"The following benchmarks show 4096x4096 SGEMM on three setups:\n\n| Implementation | Hardware | TFLOPS | % of Peak |\n| --- | --- | --- | --- |\n| cuBLAS | A100 80GB | 17.8 | 91% |\n| Custom CUDA kernel | A100 80GB | 14.7 | 75% |\n| rocBLAS | MI250X | 15.2 | 88% |\n| Custom HIP kernel | MI250X | 11.9 | 69% |\n| Vulkan Compute (GLSL) | RTX 4090 | 9.3 | 56% |\n| Vulkan Compute (GLSL) | RX 7900 XTX | 7.1 | 52% |\n\nThe vendor libraries (cuBLAS, rocBLAS) are doing assembly-level tricks that are impractical to replicate in high-level code, things like software pipelining of shared memory loads and explicit register allocation. But getting 75% of peak in portable C++ is respectable and often sufficient for custom kernels where cuBLAS doesn't have an exact routine."},{"heading":"Profiling: Where the Time Actually Goes","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#profiling-where-the-time-actually-goes","content":"The single most valuable skill in GPU programming is reading a profiler output. For the CUDA kernel, Nsight Compute reports:\n\n```\nKernel: sgemm_optimized\nDuration: 2.31 ms\nTheoretical Occupancy: 75%\nAchieved Occupancy: 71.2%\nSM Throughput: 87.3%\nMemory Throughput: 62.1% <-- Not memory bound\nCompute Throughput: 87.3% <-- Compute bound\n\nL1/TEX Hit Rate: 94.7%\nL2 Hit Rate: 78.3%\nShared Memory Bank Conflicts: 0 <-- The padding worked\n\nWarp Stall Reasons:\n stall_mio_throttle: 23.1% <-- Shared memory bandwidth\n stall_math_pipe_throttle: 18.7% <-- Good! ALU is saturated\n stall_short_scoreboard: 15.2% <-- Waiting for MIO results\n stall_not_selected: 12.4% <-- Scheduler couldn't pick this warp\n```\n\nThat ` stall_mio_throttle ` at 23.1% indicates shared memory bandwidth is the next bottleneck, the kernel loads from shared memory faster than the hardware can serve. The fix would be to increase the register tile size (TM/TN) to do more computation per shared memory load, at the cost of higher register pressure and potentially lower occupancy.\n\nThis is the game. Every optimization creates a new bottleneck. The work is done when the profiler shows a limitation by something fundamental, ALU throughput, memory bandwidth, or occupancy limits from register pressure."},{"heading":"Decision Framework","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#decision-framework","content":"An honest assessment of all three ecosystems after extensive use:\n\n**CUDA** has the best tooling by far. Nsight Compute, Nsight Systems, cuda-memcheck, the debugging and profiling story is unmatched. If you're on NVIDIA hardware, there's no reason to use anything else for compute.\n\n**ROCm** has improved dramatically but still feels 2 years behind CUDA in tooling. The ` rocprof ` profiler is usable but nowhere near Nsight Compute. The HIP language itself is 98% source-compatible with CUDA though, which makes porting straightforward. Worth it if you have AMD hardware or need to avoid NVIDIA lock-in.\n\n**Vulkan Compute** is the right choice when you need one codebase running on Intel, NVIDIA, AMD, and even mobile GPUs. The trade-off is less control over hardware specifics and a more verbose API. For ML inference engines that need to run everywhere, it's increasingly compelling.\n\nSelect based on project constraints, not vendor loyalty. Profile before optimizing."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#questions","content":""},{"heading":"When should I use CUDA vs ROCm vs Vulkan Compute?","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#question-1","content":"CUDA if you're NVIDIA-only and want the best tooling and ecosystem. ROCm if you're targeting AMD GPUs (especially MI250/MI300 for ML workloads). Vulkan Compute if you need cross-vendor portability or you're doing GPU compute in a context that already uses Vulkan for graphics."},{"heading":"What is a warp divergence and why does it kill performance?","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#question-2","content":"A warp (32 threads on NVIDIA, 64 on AMD as a 'wavefront') executes in lockstep. When threads in the same warp take different branches in an if/else, the hardware serializes both paths, threads not on the active path are masked off but still consume cycles. This can halve your throughput or worse."},{"heading":"How do shared memory bank conflicts affect performance?","level":3,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#question-3","content":"Shared memory is divided into 32 banks. When multiple threads in a warp access different addresses that map to the same bank simultaneously, the accesses are serialized. A full 32-way bank conflict means 32x slower access. Padding your shared memory arrays by 1 element per row often eliminates these conflicts."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#article-tips","content":"1. **Tip:** Profile with ` ncu --set full ` (Nsight Compute) before touching your kernel code. Teams often spend days 'optimizing' a kernel that was memory-bound, adding more arithmetic instructions that the scheduler just hid behind memory latency anyway. Know your bottleneck first.\n2. **Tip:** On AMD GPUs, the wavefront size is 64 (not 32 like CUDA warps). If you port CUDA code to ROCm with ` hipify-perl `, audit every ` __shfl_sync ` and warp-level primitive manually, the automatic translation gets the syntax right but the semantics wrong for algorithms that depend on warp size.\n3. **Tip:** For Vulkan Compute, always set ` VK_KHR_shader_non_semantic_info ` in your device extensions and compile GLSL with ` -g ` flag. Without it, RenderDoc shows your compute shader as an opaque blob and debugging becomes guesswork.\n\n- cuda\n- rocm\n- vulkan-compute\n- gpu-programming\n- matrix-multiplication\n- warp-scheduling\n- shared-memory\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Lucio Durán","url":"https://lucioduran.com/"},{"text":"Español","url":"https://lucioduran.com/es/blog/gpu-programming-cuda-vs-rocm-vulkan-compute"},{"text":"CUDA","url":"https://developer.nvidia.com/cuda-toolkit"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com/"},{"@type":"WebSite","@id":"https://lucioduran.com/#website","url":"https://lucioduran.com/","name":"Lucio Durán","inLanguage":["en","es"],"publisher":{"@id":"https://lucioduran.com/#person"}},{"@type":"WebPage","@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#page","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute","name":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication | Lucio Durán","description":"Warp divergence, shared memory bank conflicts, occupancy tuning, and an optimized matrix multiplication kernel with benchmarks across NVIDIA, AMD, and Vulkan Compute.","inLanguage":"en","author":{"@id":"https://lucioduran.com/#person"},"publisher":{"@id":"https://lucioduran.com/#person"},"isPartOf":{"@id":"https://lucioduran.com/#website"},"isAccessibleForFree":true,"datePublished":"2026-03-06","mainEntity":{"@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#article"},"breadcrumb":{"@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#breadcrumb"}},{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#article","mainEntityOfPage":{"@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#page"},"headline":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication","description":"Warp divergence, shared memory bank conflicts, occupancy tuning, and an optimized matrix multiplication kernel with benchmarks across NVIDIA, AMD, and Vulkan Compute.","datePublished":"2026-03-06","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@id":"https://lucioduran.com/#person"},"url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"GPU Computing","keywords":"cuda, rocm, vulkan-compute, gpu-programming, matrix-multiplication, warp-scheduling, shared-memory","wordCount":2283,"timeRequired":"PT12M","workTranslation":[{"@type":"BlogPosting","@id":"https://lucioduran.com/es/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#article","inLanguage":"es","url":"https://lucioduran.com/es/blog/gpu-programming-cuda-vs-rocm-vulkan-compute"}],"isAccessibleForFree":true},{"@type":"Blog","@id":"https://lucioduran.com/blog#blog","url":"https://lucioduran.com/blog","name":"Blog","inLanguage":"en","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com/"},"publisher":{"@id":"https://lucioduran.com/#person"},"isPartOf":{"@id":"https://lucioduran.com/#website"}},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"GPU Programming: CUDA vs ROCm vs Vulkan Compute, Warp Scheduling, Shared Memory, and Matrix Multiplication","item":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute"}],"@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#breadcrumb"},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"When should I use CUDA vs ROCm vs Vulkan Compute?","acceptedAnswer":{"@type":"Answer","text":"CUDA if you're NVIDIA-only and want the best tooling and ecosystem. ROCm if you're targeting AMD GPUs (especially MI250/MI300 for ML workloads). Vulkan Compute if you need cross-vendor portability or you're doing GPU compute in a context that already uses Vulkan for graphics."}},{"@type":"Question","name":"What is a warp divergence and why does it kill performance?","acceptedAnswer":{"@type":"Answer","text":"A warp (32 threads on NVIDIA, 64 on AMD as a 'wavefront') executes in lockstep. When threads in the same warp take different branches in an if/else, the hardware serializes both paths, threads not on the active path are masked off but still consume cycles. This can halve your throughput or worse."}},{"@type":"Question","name":"How do shared memory bank conflicts affect performance?","acceptedAnswer":{"@type":"Answer","text":"Shared memory is divided into 32 banks. When multiple threads in a warp access different addresses that map to the same bank simultaneously, the accesses are serialized. A full 32-way bank conflict means 32x slower access. Padding your shared memory arrays by 1 element per row often eliminates these conflicts."}}],"@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#questions","url":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#questions","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute#page"},"author":{"@id":"https://lucioduran.com/#person"}}]}],"contentHash":"2b4cca01a8736061273e4a71a727282106cfe10241b08f0a5a9ab4fd443c579d","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/gpu-programming-cuda-vs-rocm-vulkan-compute","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":77562,"contentBytes":16151}}