I don't think you can extrapolate people using cash as particularly disadvantaged. They are just as likely to be grifters using it as an excuse to avoid payment. I have seen it myself where grifters try to use - "oh, I only have a £50 note - oh, you can't change that? Best I ride free then".
Perhaps it is better in England (although not from what I see in Avanti with all sorts of chances and grifters trying to ride in First Class without paying) but in Scotland there is a real problem. This policy was brought in to reduce the estimated £11million in lost revenue. It is far better to deal with that than pander to the "cash" users.
I travel with ScotRail frequently, and perhaps I view society differently than you, but most "grifters" I see just stonewall the inspector and get off at an unstaffed station, taking advantage of the woeful state of the BTP and the unwillingness of staff to hold services for one miscreant travelling from Cardonald to Bishopton. The minimum fare would probably not stop him.
The minimum fare is moreso intended to catch out people who may take part in spontaneous fare evasion, for example by getting on a train at Paisley Gilmour Street without a ticket, and asking at the barriers for a single from Cardonald. In this case, you would now be charged the Minimum Fare (single or day return) - you would therefore be better off buying the ticket in the first instance at Paisley.
To stop people using cash as an excuse from stations where such fare evasion is common, Promise to Pay Notices were introduced, where you must select the fare (including destination, railcard, route, class and type) before travelling. Payment would then be expected in full. The notices are valid for 4 hours from time of issue, and should be exchanged at first realistic opportunity (you cannot reasonably expect all cash users to run to the inspector, this would probably violate the Equality Act amongst several others that other forum members will be well versed in).