...rather than needing a passing loop/ a tram line/ a second train in operation/ some complicated link to the MML etc (or downgrading it to just an hourly service) - what is stopping the route becoming a half hourly service?
It's 6m30ch (according to realtimetrains), so a half hourly service would only require the unit to travel around twenty six miles in an hour. There are five intermediate stops, but is the infrastructure really so poor that it couldn't cope with something faster than a unit taking around twenty minutes to cover around six miles? Needing a second train/ passing loop/ trams seems quite extreme when you consider how lax the timetable appears to be.
Maybe there are well known reasons why it'd be impossible to get much above an average speed of around twenty miles per hour, but it seems a bit "tame"!
How long a turnaround do you want at each end? I'd guess for a reliable service you'd want about 5 minutes. That means the train has to go end-to-end calling at the 5 intermediate stops in 10 minutes flat. You'd have to accelerate and go
very fast between the stops to achieve that.
Maybe you could put up with a 3 minute turnaround? That gives you 12 minutes to do the route. A bit better but still sounds iffy to me - and allowing for the driver to change ends, I suspect that would be too small a turnaround for a reliable service.
--- old post above --- --- new post below ---
As it is, I don't see a compelling reason for retaining the branch as heavy rail. Customers are disincentivised from travelling to London with the change required (and as you mention, the immemorable timetable). Very few Virgin services stop at Watford Junction either, so northbound journeys are inconvenient and you need at least two changes from St Albans to reach most northern cities on the WCML.
I would expect that once HS2 is built, Watford Junction would see many more stopping services to the North, which may make it more attractive as an interchange (especially once the Met extension is running there).
Running through trains to London from the Abbey line hits the twin problems of conflicting moves at Watford Junction and lack of spare paths to Euston, so I can't see that being viable at the moment. Maybe if the suggested link from CrossRail to Tring happens, that might relieve the pressure to Euston a little and make through running more viable.
The other problem is - as others have pointed out - getting to Watford High Street, where the shops are, isn't that easy. What would really help there is a turn-up-and-go frequency Junction-High St service. The Met line extension will help that somewhat - but only to a limited extent because of the mismatch between that 20 min Euston-Watford frequency and the planned 15 minute Met frequency. If TfL saw fit to make the WFJ-Euston services every 15 minutes so they interleaved well with the Met service, I imagine that would help massively (and it would be consistent with TfL's aim to provide turn-up-and-go frequencies in more places in London).
. 
) you'd have to move the preserved South signal box out of the way! 
) so if you buy 3 returns from the TVM at the Junction, it's £12.80 each, and if you buy 3 returns from the window, with Groupsave it's £12 for all three!