Nationwide GSMR outage 19/05

[CR165022]

So, there seems to be something slightly weird going on currently. Services seem to be stopping all along the Chiltern network?

I noticed 2C40 stopped at Aylesbury at first and just assumed that maybe the unit broke down and was being swapped, but then noticed 2H35 from High Wycombe not going either. Now, little to no services seem to be moving, 1H40 & 1T27 are stopped at Haddenham & Thame Parkway, 1Y35 & 1R29 are stopped at Saunderton and 2W29 & 1H33 are stopped at Wembley Stadium.

 

[150219]

Are you asking a question?

 

[CR165022]

Yes, sorry should have specified, I'm just wondering if anyone knows what's going on because it seems out of the ordinary.

 

[LittleRedTrain]

From Chiltern JourneyCheck

Chiltern Railways JourneyCheck - Train times and live real time delay/cancellation/disruption information.

Chiltern Railways JourneyCheck provides real time, up-to-date train times and train disruption information keeping you informed about any delays or cancellations on the network and helping you know what to do when your journey is affected. Train Cancellations, Delays, Alterations and...

www.journeycheck.com

Delays to services between Aylesbury Vale Parkway and London Marylebone


Due to a fault with the radio system between the driver and the signaller between Aylesbury Vale Parkway and London Marylebone all lines are blocked

 

[TheLastMinute]

GSMR playing up again across a wide area. GWR also being affected.

From https://www.journeycheck.com/greatwesternrailway/

Cancellations to services on all routes

Due to a fault with the signalling system trains have to run at reduced speed.
Train services running across the whole Great Western Railway network may be cancelled or delayed. Disruption is expected until 15:00 19/05.

Customer Advice
-
What has happened?
-
We have had a nationwide GSMR outage
-
What are we doing about it?
-
We are working with Network Rail to implement a solution.

 

[DelW]

Is this a repeat of Sunday's problem?

What’s going on on the Chiltern Mainline today (17/05)?

My brother is currently trying to travel today and his train 1Y29 has apparently been cancelled due to “some sort of comms fault”. This train last reported on RTT at Oxford Canal Jn, and on Traksy there is a berth saying “118” in that area. From Aynho/Oxford up to Marylebone, there are basically...

www.railforums.co.uk

The issue was fixed at around 1508 on Sunday. In essence, no information from the Marylebone signalling panels was leaving to the outside world, though the signalling system itself was working fine. The backlog of train movement information seems to have been stored and released to Trust (but only at Network Rail monitoring points - not all calling stations). Real Time trains reflects this, though Darwin which feeds station screens and all industry related websites and Apps has not backloaded the information to before 1508. All public facing systems were accurate from the next time any individual train passed a monitoring point

Hoping it isn't a repeat of Thursday 7th May's comms problem, which caused widespread chaos across south east England.

 

[43095john]

Reports coming in that the GSM-R Radio system has gone down with operators reporting issues all over. Presumed a nationwide outage

Most services being limited to 100mph max as per rulebook but expect issues for remainder of the day

 

[The Planner]

Reports coming in that the GSM-R Radio system has gone down with operators reporting issues all over. Presumed a nationwide outage

Most services being limited to 100mph max as per rulebook but expect issues for remainder of the day

100mph for the next 4 hours.

 

[TheLastMinute]

Is this a repeat of Sunday's problem?

What’s going on on the Chiltern Mainline today (17/05)?

My brother is currently trying to travel today and his train 1Y29 has apparently been cancelled due to “some sort of comms fault”. This train last reported on RTT at Oxford Canal Jn, and on Traksy there is a berth saying “118” in that area. From Aynho/Oxford up to Marylebone, there are basically...

www.railforums.co.uk

I'm unsure what the issue was Sunday, but reading the linked thread, it appears to have been limited to just Marylebone IECC. Today's problem is GSMR related, I'm currently seeing reports of it being over a very large area, maybe nationally. Very similar to what happened on 7th May, but that's not to say it's necessarily related - it could well be a completely different failure in the GSMR system.

Thread from 7/5: https://www.railforums.co.uk/thread...-due-to-radio-fault-07-05.301898/post-7776084

Seem to be stuck in a GSM-R failure on Wessex. Some services have adjacent berths marked with 'GSMR'. Looks pretty widespread, maybe a backbone network issue?
Interesting to see what the root cause is!

 

[orestad]

Currently affecting Avanti, GWR, Hull, LNER, Lumo, Southern and WMT too. Unlucky that Southern’s got the same issues as last time.

 

[NSEWonderer]

Yep SWR too

 

[Horizon22]

It’s nationwide. Every operator is impacted to some extent. It’s going to be a long day.

 

[43095john]

GSM-R reported to have come back online approx 1300 in some areas so things should slowly return to normal

 

[wanderingriver]

Like with the last GSM-R outage, there is also a Three Mobile and o2 outage that started at the same time. That appears to be just parts of South London though

 

[Trainspecial]

Can confirm that there are delays. A significant number of departures from Exeter Central are showing as "Delayed"

 

[Roy Badami]

There's serious disruption on the Northern City Line, which is unsurprising given ETCS relies on GSM-R.

I'm curious, though - what happens on the Northern City Line when ETCS is unavailable? Can they use conventional signalling (like on the Thameslink core) or some other form of degraded working? Or is/was the line completely suspended until ETCS comes back on line?

 

[zwk500]

There's serious disruption on the Northern City Line, which is unsurprising given ETCS relies on GSM-R.

I'm curious, though - what happens on the Northern City Line when ETCS is unavailable? Can they use conventional signalling (like on the Thameslink core) or some other form of degraded working? Or is/was the line completely suspended until ETCS comes back on line?

It's no-signals, so degraded working. Depending on the Rules, On-Sight or Staff Responsible Mode may be offered, but given the nature of the tunnels it may be that once trains have got to a platform they don't leave it.

Are the tunnel telephones still in place?

 

[185143]

It's no-signals, so degraded working. Depending on the Rules, On-Sight or Staff Responsible Mode may be offered, but given the nature of the tunnels it may be that once trains have got to a platform they don't leave it.

Are the tunnel telephones still in place?

I've passed signals at danger under degraded working on Merseyrail before. But only one at a time, rather than (say) the whole loop line.

 

[sh24]

At Paddington right now. Delays on the board (13:15 to Cardiff is still here, random Elizabeth line service to Reading showing on main platforms) though a pair of 387’s have just arrived so some things are moving.

Think I’ll postpone my meeting in Oxford and mini shack bash on the Cotswold line. Not worth the potential hassle!

 

[BazingaTribe]

GSMR playing up again across a wide area. GWR also being affected.

From https://www.journeycheck.com/greatwesternrailway/

Can confirm -- the 12.40 to Cardiff just left Reading with me on it at 13.36, and is terminating at Newport.

And there was a fire near Brockenhurst.

Oh well, I've rolled the dice enough this year that snake eyes had to come up at some point.

 

[ATS3]

What's causing all of these outages, is it just coincidence?

 

[Horizon22]

Seemingly back on line now. But the disruption will continue for a good few hours.

What's causing all of these outages, is it just coincidence?

They won’t know that yet before an investigation.

 

[trundlewagon]

Not sure if it was coincidental but all the signalling went to pot in the New Street area at the same time as the GSMR started playing up.

 

[zwk500]

Not sure if it was coincidental but all the signalling went to pot in the New Street area at the same time as the GSMR started playing up.

As in going blank or showing wrong-side failure aspects?

 

[cduncan]

Doesn't look as bad as last time on SWR. Suburban looks okay but did hear some disruption on some long distance services eg 1620 Waterloo to Exeter starting from Basingstoke.

 

[DelW]

Doesn't look as bad as last time on SWR. Suburban looks okay but did hear some disruption on some long distance services eg 1620 Waterloo to Exeter starting from Basingstoke.

Indeed, Waterloo seems to be running normally at the moment. 1520 to Exeter has just left on time.

 

[357]

I was stopped approaching Newark North Gate. SG didn't work and the GSMR wouldn't connect a call. The SPT wasn't working either.

After a while the SG worked, I got a Wait back. The signal then cleared and I continued north about half an hour late.

Nobody ever told me to only do 100mph.

 

[BazingaTribe]

Got into Cardiff ten minutes late -- not a big deal, but I did get a delayed earlier train and we had to change at Newport, so no big deal.

Just before our train left Basingstoke I thought I'd be sat there several hours. GWR were pretty good about jumping on a green signal when they had the chance ahead of a Cross Country service.

 

[TheLastMinute]

I'd imagine the delays were less this week as most were already familiar with the contingency following the similar failure on the 7th, e.g. broadcasting messages to train crews about 100mph limit and authorising use of guard's mobile phones vice GSMR.

 

[trundlewagon]

As in going blank or showing wrong-side failure aspects?

A load of signals dropped back to red in front of trains, so nothing wrong-side (as far as I'm aware) but the SPTs weren't working either. Then after about 15 minutes signals started clearing, phones started working and things started moving again.