interviewer hands you a recurrence: roughly alpha * old_attention + (1 - alpha) * new_attention. implement it, vectorize, discuss memory / perf. vectorization was where they pushed. naive python loop -> numpy -> discuss why a fused kernel would be faster -> hand wave a triton impl

Nvidia decaying attention

interviewer hands you a recurrence: roughly alpha * old_attention + (1 - alpha) * new_attention.