This paper introduces mutation analysis to measure whether GPU-kernel benchmark correctness checkers — which now feed leaderboards and RL reward signals — are actually adequate, by injecting over 10,000 deterministic faults into verified CUDA kernels and scoring test protocols by the fraction they detect. It finds the official KernelBench checker misses one in six witnessed faults, with precision-related faults escaping detection 78.6% of the time versus 8.7% for arithmetic faults, and shows a published fuzzing recipe wrongly rejects correct kernels 107 times. Optimizing test suites over the resulting fault matrix reaches 98% detection with just two inputs per problem, and the released suite lets others audit their own checkers.
