People who do not seem to understand that language is different than they wish it to be, are the actual morons. Not only morons, but pampas morons. Language is messy, imprecise, and always in flux. Language is a construct of the collective of its speakers, not you alone, nor anyone else. This is why we have specific lexicons for various industries, and academic fields. Even those are constantly being updated, and revised.

You hate on people that use literally this way, but you do the same thing yourself...

Moron is a term once used in psychology and psychiatry to denote mild intellectual disability. The term was closely tied with the American eugenics movement. Once the term became popularized, it fell out of use by the psychological community, as it was used more commonly as an insult than as a psychological term. It is similar to imbecile and idiot.

Wikipedia

But unless the people that use "literally" in the colloquial sense you are actually using a term that is tied to eugenics and the idea that disabled people are inferior. Maybe you should have thought about the words that come out of mouth?

The use of literal to mean figurative has been common for hundreds of years (literally). If it's good enough for James Joyce, it's good enough for you