I presume banks have reciprocal arrangements and don't actually charge each other - but the private ones do as there's an obvious cost to provide a machine, and refill it.
But, £2 is too much in my opinion and I'll vote with my feet! If it was 50p or less, I'd probably consider that a suitable fee for the convenience of not having to walk further to a free one.
Not entirely sure how it works, but I guess the major banks see that allowing people to access their money from any cash machine at any bank for free is better than charging for the service. Sure a few years ago that some banks were going to charge for withdrawals from their machines but using a different bank's card, but this quickly went out the window.