It's definitely a problem. I am not sure there are any easy answers either, because when things go badly wrong there's going to be confusion and limited options to deal with them.
A lot of the problems are solved by not advertising an expected time until a certain threshold of confidence is reached:
E.g.
1. Have fleet confirmed the next working of the inbound, with or without set swaps?
2. Have crewing confirmed the outbound driver is available?
3. Has the inbound train passed the problem area?
4. is there anything stuck in front of it that is likely to slow it down significantly?
Only at point 3 do you advertise an 'expected time' on the board, and consider point 4 when you do so. Points 1/2 can be confirmed in either order as required. If that means that 'boarding expected' is only advertised a few minutes before it begins, so be it.