BREAKING
LeetCUDA: 200+ GPU Kernels Open-Sourced
Kernels, Benchmarks and PyTorch Notes
Kernels Sorted by Difficulty
1
Easy
↓
2
Medium
↓
3
Hard
↓
4
Hard+
0
%
of cuBLAS TFLOPS (low)
0
%
of cuBLAS TFLOPS (high)
FlashAttention-2 Speedups on Attention
FA-2 RTX 3080
55
FA-2 baseline
37
L20 ffpa-attn
104
SDPA baseline
58
Gains Vary by Size and Device
AI NEWS BLITZ
An open-source project called LeetCUDA is bundling over 200 hand-written CUDA kernels for GPU engineers.