The term turned negative because it represented something that no one wanted to be associated with. Even if you were to use the phrase "unicorn-blessed" in the same context, over time, it could also develop a negative connotation. It's not the terminology that's a problem, but the situation it describes.
"Idiot", "moron", and "imbecile" used to be medical terms.
Today, one often hears "He is special needs" said by people who think they are being politically correct. But that's awfully wrong. He might have special needs, but he is not needs. He is a person. To me, that's a lot more dehumanizing than saying "retarded".
Then they'll say "but you know what I mean". That's the point, exactly.
The ultimate culmination being certain subreddits where the banned "retarded" has been replaced by the homonym "regarded", such that referring to someone as "highly regarded" is understood to mean the former, not the latter.
At some point we may collectively realize that controlling language in such a way is impossible. People will just find alternative ways to express exactly the same sentiment.
"Idiot", "moron", and "imbecile" used to be medical terms.