applied for a GPU computing role. the interview was intense: 1. wrote a CUDA matrix multiplication kernel from memory. then optimized it with shared memory tiling 2. discussed memory coalescing patterns for different access patterns 3. system design: design a distributed training pipeline for a 175B parameter model 4. deep dive into my past work on inference optimization this is the most specialized interview ive ever done. if you dont live and breathe GPU computing you will not pass

Nvidia CUDA engineer - they asked me to optimize a kernel on a whiteboard

1daa6194

applied for a GPU computing role.

29bc31e3

nvidia interviews are basically a PhD defense for GPU programming. the bar is extremely high and extremely narrow

d5683f6c

the distributed training pipeline question - did they ask about model parallelism vs data parallelism vs pipeline parallelism? thats the key design decision they want to explore

8a493535

nvidia TC for specialized roles is insane right now. senior GPU engineers are getting 450-600k because the supply is so limited