Oh absolutely. I just remember the same member of staff trying to stand in my way and telling me "standard class is that way" when I tried to enter the carriage - so it wasn't just a few frustrated words. Thankfully I haven't experienced anything like that since (we're talking over seven years ago).
Back on topic, yes a public-facing role does grind you down - I had my fair share of abuse, and was more than happy to have a rant about it out of passenger's earshot.