Epsom Downs and Tattenham Corner are far better for accessing the racecourse, and are already fully integrated within the London Zones and Oyster system. Indeed, I would suggest this may have been part of the incentive for the Oyster rollout to the Tattenham Corner branch, which is very much on the boundary of Greater London. Epsom Downs is somewhat more understandable in these terms, as it is only a short distance from Sutton, which is part of a conurbation closely connected with the South London rail infrastructure.
The real issue with Epsom itself is that both Ewell East and Ewell West are within the Zones on the lines into the station, but somewhat illogically the Oyster system has not been extended one station further to link up the lines and provide a convenient boundary for many passengers. I am sure Oyster would be much better used - and used more frequently in the correct manner - if Epsom was included, and I understand it will be. I don't see that there is any real need to extend it much further, if at all - certainly no further than Dorking, which is the local boundary of TfL-branded buses.
If only the boundaries for zones could be:
Epsom - Zone 6/7
Ashtead/Leatherhead - Zone 7
Box Hill/Dorking Stations - Zone 8
Betchworth - Zone 9 (to allow travel on Oyster via Redhill or Dorking)
Reigate/Salfords - Zone 7/8
Redhill/Merstham/Earlswood/Nutfield - Zone 7
Horley/Gatwick Airport - Zone 8
Three Bridges/Godstone/Edenbridge Stations - Zone 9
Hurst Green/Oxted - Zone 8
Woldingham - Zone 7
Such would seem to be fairly clear-cut and well-defined.