double dissent

From many to one

In the last post we used GPUs to add two vectors, element by element. It’s what we call an embarrassingly parallel problem, but you might be thinking: is this all that GPUs are suited for? Doing independent things in parallel with no coordination? One operation often arising in LLM inference is summing all elements of a vector. On a CPU, you would do this: How would you parallelize this? Each valu…