I just instruct my guys to say "I have to prioritize what I'm doing right now. Go talk to HeyLaughingBoy if you need me to work on something else."

They have no problem doing that. Do it often enough and the people learn to just come straight to me instead.