Rugby ROC Evacuated - 18/09/18 Evening

[edwin_m]

That was just the hype on the glossy brochure until the reality kicked in.

If every signalling workstation in every ROC was maxed out to the amount of railway and work each signaller is controlling, all at 100% with normal running. Then if every ROC is designed for the maximum amount of workstations just to control its area based on the above. So now we are where the dream hits reality as if the signaller is working at 100% capacity under normal running, then what happens in the times of failure. There is now more workload than a idvidual can safely handle, you can’t pass some workload to their colleagues as they are also running at 100% under normal running plus working extra from the fallout from your problem. So now the areas of control has shrunk so each signallers is working around 70% or capacity to allow for working in times of disruption. This has had the knock on effect that you now need more workstations in buildings that weren’t designed to house that many, hence the mini or sub ROC a concept has been touted.

Also if ROCs are full of workstations controlling their own part of the railway network, where do you put say 25% extra workstations to control say the largest ROC (York I beleive) into say 4 other ROCs taking equal share of the workload. There are at least 7 types (or sub types) of signalling workstations that I’m aware of out there from the various signalling manufacturers each which requires training on, all local rules per each location/workstation is unique to the location. The generic rule book does not cover everything. If you do find space to instal these extra workstations, will they mirror the same type of the ROC who’s area is being taken over or will it be of the same type already installed in the ROC which is taking the extra workload and does that pose any signalling control risk. When do you put this over capacity into each ROC, in stages keeping pace with the migration project or once its finally completed in many decades time.

But the big issue is who will man theses dust covered workstations, will it be the staff from the effected ROC after being put up in local hotels and how long will the railway shutdown be until they can temporarily relocate. Or are they going to go to the government and say that despite the ROC concept being partially about signalling staff reducuctions that they will need to employ a pool of high grade signallers to sit aound in ROCs basically doing nothing to act as standby contingency just in case there is a rare occurrence of losing a complete ROC. Who will maintain their competency and how often would they get to actually control real trains to keep their skills and knowledge up?

This summarises the issue very well. A changeover would be possible in hardware/software terms but when you start considering the human factors it gets a lot more difficult.

Of course rather like rail accidents versus road accidents, conventional signalling centres and signalboxes have a multitude of small faults that don't usually hit the headlines. They probably take longer to fix because they are less likely to have technicians based there. I don't suppose anyone has tried to calculate whether the disruption from these faults exceeds that from the fewer but larger problems that occur in ROCs.

 

[snowball]

That was the big promise when the magical ROC concept was planned, if Rubgy went down then say West Midlands could take over at the flick of a switch.
.

Isn't everything at West Midlands supposed to be recontrolled to Rugby in due course? West Midlands isn't on the lists of eventual ROCs that I've seen ... though maybe they're backtracking the plans a bit now?

 

[Stampy]

Ah so the SCC is the very bland building that has Rugby SCC on the side and is the ROC the large green building with Network Rail on it?

From a photo I took a while ago.....

 

[The Planner]

Isn't everything at West Midlands supposed to be recontrolled to Rugby in due course? West Midlands isn't on the lists of eventual ROCs that I've seen ... though maybe they're backtracking the plans a bit now?

WMSCC isn't going anywhere, in the background I think they have realised how much it would cost.

 

[1e10]

oops!
ALL critical infrastructure needs work-arounds built in.
just think of the internet and your packets of data being transported around.

If a node/hub goes down there is a network of other hubs/nodes that are in place to redirect traffic.

I would suggest that the rail industry needs to get up to speed on this pronto, otherwise they will find full automation upon them rather quickly

I "run the internet" for a day job as an engineer, working with data centres and ISP backbones and can assure you the industry is not without problems.

Your analogy is correct in some respects; networks often run such that if a single piece of equipment fails, something else will take over. However I have seen firsthand (with my own employers and working with suppliers) the fallout that happens when a critical data centre site suffers a fault of some description significant enough to take the physical data centre site offline.

The problem is that sometimes resilience comes with issues of its own. Got two generators that operate in "parallel", sounds fantastic until they have an issue syncing with each other and both trip out! Got multiple UPS units working in parallel? Sounds fantastic until a firmware issue arrises and takes the lot out.

Back to the topic at hand. What geographical area does Rugby ROC cover? Is there a list anywhere of all ROCs operating in the country?

 

[snowball]

Back to the topic at hand. What geographical area does Rugby ROC cover? Is there a list anywhere of all ROCs operating in the country?

I gather from posts upthread that at present it only controls the Stafford area?

The original plan was that it would eventually cover the whole southern half of the LNW "Route", including from Euston to somewhere approaching Crewe and including the whole Birmingham area, but clearly from #34 that's no longer the plan.

 

[Tio Terry]

Surprised it was necessary to evacuate a ROC so quickly. Normally the operating floor is a 2 hour fire resisting block by design with 2 escape routes to make it pretty resilient and to allow time for an orderly shut down of services. Perhaps, because it's not fully in service, it's not operating to those rules yet, I don't know.

There have been some comments on here about controlling signalling systems from other sites, from an engineering perspective this is possible but from a practical perspective is very unlikely to happen because of the reasons already given. But please remember it's not just signalling, you also need telecommunications. You may be able to manage without SPT's if you have a reliable GSM-R system with good coverage but if the control area includes level crossings with telephones then it's going to be difficult to run a service safely. Diverting all calls to another centre has it's difficulties!

 

[noddingdonkey]

I don't see why a second set of workstations couldn't be provided on the same site, perhaps in a portakabin or similar, for such situations. I assume that the workstation itself is just a control surface for kit sitting in a server room, so having a different control surface that can be used if the primary one is unavailable should not be prohibitively expensive, given the savings by putting so many eggs in one basket.

 

[Highlandspring]

It was the smoke that was the issue I believe. On a related note I recently visited NATS at Prestwick and they were in the process of installing a large area of contingency workstations in a corner of the operating floor to be used in the event of the other en route control centre at Swanwick being ‘unavailable’. I was told that these workstations would be manned by staff from Swanwick and that a similar fall back for Prestwick was being installed at the Swanwick centre.

 

[IanXC]

I have started a new thread around the ROC migration project here: https://www.railforums.co.uk/threads/live-signalling-workstations-in-rocs.169979/

 

[carriageline]

Would be an improvement over the present TBROC service.

Care to elebarotate? Would love to know more!

 

[adrock1976]

Would be an improvement over the present TBROC service.

Could this be translated please?

It is a long standing forum policy not to use abbreviations or acronyms without first saying what they mean.

 

[snowball]

I imagine TB

Could this be translated please?

It is a long standing forum policy not to use abbreviations or acronyms without first saying what they mean.

I imagine TB means Three Bridges.

 

[DPWH]

oops!
ALL critical infrastructure needs work-arounds built in.
just think of the internet and your packets of data being transported around.

If a node/hub goes down there is a network of other hubs/nodes that are in place to redirect traffic.

I would suggest that the rail industry needs to get up to speed on this pronto, otherwise they will find full automation upon them rather quickly

I think that in case of failure of the WCML, the general plan is to use fall back onto other routes, i.e. Chiltern/MML/ECML, and other modes, mainly coaches, but perhaps taxis in a few instances. Consequently, people will be late, and might spend several hours unpleasantly wedged in a stranger's armpit, but they will get home eventually.

 

[xc170]

Surprised it was necessary to evacuate a ROC so quickly. Normally the operating floor is a 2 hour fire resisting block by design with 2 escape routes to make it pretty resilient and to allow time for an orderly shut down of services. Perhaps, because it's not fully in service, it's not operating to those rules yet, I don't know.

Great in theory but would you continue to work in a building you know is on fire? Especially in the wake of Grenfell...

 

[Tio Terry]

Great in theory but would you continue to work in a building you know is on fire? Especially in the wake of Grenfell...

I don't think you can equate a multistory multi-occupancy building with a ROC, as far as I am aware they are only two story buildings and only occupied by NR staff who will be adequately trained for cases of evacuation. Would I work in one? I've spent many, many, years working in railway buildings in similar circumstances. "Double knock" fire alarm systems in places like Waterloo General Offices mean you don't evacuate on the first alarm call but wait for the second, so I'm sure there are plenty of us that have first hand experience of continuing working after the first alarm has been raised.

 

[Tio Terry]

It was the smoke that was the issue I believe.

Interesting. Makes me wonder about the design - or maybe because it's not fully occupied the full facilities are not in use. It's usual for the design of the air con system to be such as to contain smoke within a compartmentalised area and not let it spread. Maybe something combustible was in the wrong place? Anyone know what the smoke came from and what caused it? Was it carried in to the operating floor by the air con system from outside the building?

 

[mark-h]

On a related note I recently visited NATS at Prestwick and they were in the process of installing a large area of contingency workstations in a corner of the operating floor to be used in the event of the other en route control centre at Swanwick being ‘unavailable’. I was told that these workstations would be manned by staff from Swanwick and that a similar fall back for Prestwick was being installed at the Swanwick centre.

For more resilience it would be best to have a contingency centre located near to each centre (far enough away to not be affected by the same issues as the main centre). This would allow controllers/signallers to get to the substitute quickly (NATS Prestwick to NATS Swanwick is over 450 miles by road- [air travel will not be available!]).

Having each centre split into two twin sites with each able to cover the other with staff trained on the work of their sister site would be even better.

 

[gsnedders]

I "run the internet" for a day job as an engineer, working with data centres and ISP backbones and can assure you the industry is not without problems.

Your analogy is correct in some respects; networks often run such that if a single piece of equipment fails, something else will take over. However I have seen firsthand (with my own employers and working with suppliers) the fallout that happens when a critical data centre site suffers a fault of some description significant enough to take the physical data centre site offline.

The problem is that sometimes resilience comes with issues of its own. Got two generators that operate in "parallel", sounds fantastic until they have an issue syncing with each other and both trip out! Got multiple UPS units working in parallel? Sounds fantastic until a firmware issue arrises and takes the lot out.

Back to the topic at hand. What geographical area does Rugby ROC cover? Is there a list anywhere of all ROCs operating in the country?

And there's also plenty of cases where the failover is simply never tested, and gets broken by something that seems sensible at the time. I've worked on systems where we regularly did failover tests and ones where we didn't; almost always, the ones that worked well when **** hit the fan were the ones which were tested.

For more resilience it would be best to have a contingency centre located near to each centre (far enough away to not be affected by the same issues as the main centre). This would allow controllers/signallers to get to the substitute quickly (NATS Prestwick to NATS Swanwick is over 450 miles by road- [air travel will not be available!]).

Having each centre split into two twin sites with each able to cover the other with staff trained on the work of their sister site would be even better.

Air travel would be available; bear in mind that there is always a fallback, because planes can't just stop moving. It's likely in general flights wouldn't be cleared to take-off as long as the other site was dealing with the full area, but exceptions can and are made.

 

[Tin Rocket]

Interesting. Makes me wonder about the design - or maybe because it's not fully occupied the full facilities are not in use. It's usual for the design of the air con system to be such as to contain smoke within a compartmentalised area and not let it spread. Maybe something combustible was in the wrong place? Anyone know what the smoke came from and what caused it? Was it carried in to the operating floor by the air con system from outside the building?

An attempt to toast a Doorstep Cheese and Ham sandwich which was wedged into a too small toaster slot.

 

[Deafdoggie]

I don't think you can equate a multistory multi-occupancy building with a ROC, as far as I am aware they are only two story buildings and only occupied by NR staff who will be adequately trained for cases of evacuation. Would I work in one? I've spent many, many, years working in railway buildings in similar circumstances. "Double knock" fire alarm systems in places like Waterloo General Offices mean you don't evacuate on the first alarm call but wait for the second, so I'm sure there are plenty of us that have first hand experience of continuing working after the first alarm has been raised.

Most public builds, shopping centres, cinemas, etc work to this system too, so it really is very common. You will have been in a building with this evacuation policy I assure you. You just didn't know it!

 

[Tio Terry]

An attempt to toast a Doorstep Cheese and Ham sandwich which was wedged into a too small toaster slot.

OK. So self inflicted. Still some questions though, toaster should have been in the mess room with it's own ventilation system and fire/smoke alarm. The design of the operating floor should be such that smoke from the mess room should not be able to get to the operating floor.

 

[xc170]

So, okay, we've gone from a fire in a server room to someone burning their lunch?

Really.

 

[Skie]

Large office buildings have multi-stage evac systems. At the basic level: The floor where a fire was detected and the one above and below are given the evac signal immediately, whilst the other floors are given the prepare signal. If the alarm is not silenced after a cetain amount of time, or if another sensor/call point is activated, then the next set of floors nearest get the evac signal. It's a sensible system in modern buildings that shouldnt see rapid escalations, and stops there being thousands of people in a queue in a stairwell. A building such as a ROC would use a slightly modified setup, but there are plenty of ways of making sure a space is 'safe' from fire nearby for a certain period, especially in low-rise buildings.

Modern(ish) HVAC systems dont circulate air from one place to another, unless the building is using a really old fashioned central air system. The vents in the ceiling likely just go to an air handling unit in the space immediately above it, which gets a coolant piped to it from a heat exchanger outside. The rooms air is just recycled through that box, it doesn't go anywhere. Fresh air is pumped in via other vents, and that will all come from a well filtered intake on the roof. Those images of John McClaine crawling through air vents are, unfortunately, unlikely. If a building does use a central air system, it will be shut down automatically as soon as the fire alarm is activated.

 

[daveshah]

Most public builds, shopping centres, cinemas, etc work to this system too, so it really is very common. You will have been in a building with this evacuation policy I assure you. You just didn't know it!

Most famous of course being "Inspector Sands" on LUL and in some rail stations

 

[mark-h]

A building such as a ROC would use a slightly modified setup, but there are plenty of ways of making sure a space is 'safe' from fire nearby for a certain period, especially in low-rise buildings.

I would expect the fire service to insist on an evacuation if there is a significant fire. Post Grenfell there will be a lot more reluctance for them to give "stay put" advice.

 

[Tio Terry]

I would expect the fire service to insist on an evacuation if there is a significant fire. Post Grenfell there will be a lot more reluctance for them to give "stay put" advice.

Fire authority will have already approved the two hour fire rating of the operating floor before the building was occupied. This is not a high rise building with members of the public occupying it. The occupants are fully trained in fire precautions and evacuation requirements, and there is a much more comprehensive fire detection and management system, there is no comparison with Grenfell.

Sounds more to me like someone had the toaster on the operating floor!

 

[Sunset route]

I would expect the fire service to insist on an evacuation if there is a significant fire. Post Grenfell there will be a lot more reluctance for them to give "stay put" advice.

If there is a significant fire then the two stage alarm would go straight to stage two which is evacuate, so it’s send Emergency alarms, REC calls and evacuate the building, whilst SSM advises control, ECROs and (airport control tower if there is one).

 

[Highlandspring]

Sounds more to me like someone had the toaster on the operating floor!

Is there a source for that or is it baseless speculation? All the internal info about the evacuation has referred exclusively to an actual confirmed fire in the IT server room.

 

[mark-h]

Fire authority will have already approved the two hour fire rating of the operating floor before the building was occupied.

You have a lot more confidence in them than I do!

Besides that the fire fighter in charge would not necessary know the full specification of the building (and would know even less about what the builders have built).