This.Of course that sucks, but that still doesn't defend allowing people to continue driving when they are literally unsafe doing so or have been told not to due to medical issues, which is the point I'm making. Driving is something that can be taken away from you very quickly for very good reasons in circumstances that aren't actually that rare.
To expand slightly, there's a couple of main ways to lose a licence.
The first is medical, which, as unfortunate as it is, is fundamentally a risk of living somewhere with poor transport connections. Public transport should be better even for many rural communities, but we can't let people continue driving when medically unfit just because it's inconvenient for them to be without a car. Many medical conditions result in people having to change circumstances, and in some extreme cases that may mean moving from a lifelong home to somewhere better connected to essential services.
The second is being disqualified for one or more driving offences. I actually think the UK strikes the right balance with its points system. If you're doing for speeding (but not ridiculous speeds), you get 3 points, if not a speed awareness course. You've got 12 points to 'play with' so you'd have to be habitually speeding to lose your licence this way. If you make some non-trivial driving mistakes but are not clearly racing or driving like an utter twonk, you might get 3 points for careless driving. If you use a mobile phone while driving, 6 points. For minor offences, you are given a minor punishment which doesn't result in loss of licence. If you make repeated minor errors, or serious issues like drink/drug/dangerous driving, you are rightly banned.
People reaching a 'certain age' who live somewhere where driving is necessary ought to plan for the not-unlikely eventuality that they may be unable to drive one day. Whether that's getting food and supplies delivered, finding a local cab firm, asking friends/family to help, using public transport, or ultimately possibly moving home, are all possibilities which should be thought about. And by a 'certain age' I don't necessarily mean elderly - if you're in your late 50s and obese with heart problems, plan for the day you may be diagnosed with diabetes and suffer a severe hypo at which point you must stop driving and inform DVLA (for example).
I have a family member who would love to live very remote, but with her diabetes and liver cirrhosis it would be a terrible decision, as she's already had her licence revoked once for the diabetes and no doubt it'll happen again some time in the next few years. So she just knows she can't live that far out, and is even considering moving to somewhere nearer town so she has better bus and minicab options.