But still if I feel threatened at all, I just back off and make sure the office door is secure only coming out to dispatch a train if needed until I feel it's safe enough to leave the door open.
The problem with that is that in any sort of delay/cancellation situation, one person gets annoyed with you and you withdraw, potentially leaving a many stranded passengers on the platform with no staff giving them any indication of when/if a train is going to turn up (or how to make alternative arrangements).
Even with good PA announcements (this is a rarity - the usual is "<TOC Name> regrets to announce the cancellation of <Service> we apologise for any inconvenience caused" - repeatedly with no indication of what the problem is or when it might be resolved), there will still be passengers with queries (ranging from "I have an AP and I've missed my connection" to "If I get a taxi to <nearby station>, will my ticket be valid/are trains running there?").
Obviously, the whole situation is less than ideal for everybody involved and some passengers will unfortunately be unreasonable, but having staff members disappear as soon as somebody raises their voice only serves to make a bad situation worse.