Chances are you might derail, which would be very bad for everybody involved. I dont know whether this has just gone without saying in the thread so far but it is worth highlighting that the driver has a bit more indication than which way the points are facing as to where he is headed. There are whole sequences of signals leading up to junctions and you expect to see them show certain aspects in a specific order to send you on the route you need to take. Again, it depends on the location and the speed over the junction, but at higher speed junctions (increasingly the norm) you will get flashing yellow signals before the junction signal showing which way the route is set. If you see a green at that signal where you were expecting a flashing yellow then you know you have been wrong routed so you stop the train immediately and tell the signaller that you have been wrong routed. Same if you are being brought down to a red signal for a route to clear over a lower speed junction, if there is a green where you were expecting a yellow then again, stop immediately and tell the signaller.
I guess the easiest trap to fall into is when there is a route out of a platform set by a stencil indicator (the sort of signal which just has a letter above it, ie "M" for main or "B" for branch) above a green signal, and you can miss that if you arent careful, particularly as platform duties can be distracting. I have been wrong routed several times and, thankfully, have never taken one yet. When you think of how many trains run over how many junctions every single day then the amount of wrong routes given is absolutely tiny. It is a fraction of a fraction of a percentage point, but it still happens. Usually drivers notice and save for a minute or two delay while the correct route clears and signals time out, passengers are blissfully unaware of it.