It's Not Ok to Touch My Hair!

When my girlfriends and I get together and do our girl talk, sometimes the weirdest topics arise. One of which is "why do strangers think its OK to touch your hair?" I could understand if they ask first, but a lot of individuals really don't and I am boggled as to why. I consider my hair to be very much apart of me just like my nose or arm. To be quite frank its annoying and unfortunately I have seen people get very upset by this very "not welcoming" gesture. Is this a universal thing?