Crossrail opening delayed (opening date not yet known)

[hwl]

Obviously a pointed view, but I have to agree, at least one of the major "consultancies" (keeping on this side of the law) appointed seemed to contribute diddly-squat to the process or achieve what they originally were appointed to manage. I wouldn't use them again and suggest TfL have the same attitude.

In one case the former management team reduced the scope of one consultancies role because they wanted light touch oversight and less centralisation less than a year after contract award. This prevented lots of issues joining the "problem list" far far earlier. The original scope would have meant many awkward question being asked earlier. Crossrail hired them to acts as part of a brain but then decided they want to be an invertebrate instead!
Crossrail are still struggle to create the central brain function under the new team.

 

[Taunton]

Getting possessions on the Crossrail side to test the transition probably aren't impossible; after all they managed it with Heathrow tunnels which are at least as busy.

In a sensible project plan, the transitions between signalling systems (if you REALLY needed so many different ones) would have been identified at the start as a likely key issue, developed on a test track somewhere with all the changeovers, the software developed early in the project rather than at the last minute, and all run and tested so there was none of this early days but last minute prototyping and sudden surprises on the main project.

The issues described in this month's Modern Railways really sound like Train Software School 1.01.

 

[JonathanH]

In a sensible project plan, the transitions between signalling systems (if you REALLY needed so many different ones) would have been identified at the start as a likely key issue, developed on a test track somewhere with all the changeovers, the software developed early in the project rather than at the last minute, and all run and tested so there was none of this early days but last minute prototyping and sudden surprises on the main project.

The issues described in this month's Modern Railways really sound like Train Software School 1.01.

Doesn't that exist at Old Dalby? There seem to have been 345s there for some time and indeed 345s taken from Worksop for short periods.

There is some evidence of testing at Old Dalby here http://pearsonsengineeringservices.co.uk/railway-signalling-testing-services.php

Railway Signalling Services - Pearsons Engineering Services partnering Green Light Signalling
Signalling Projects: 2016-2017

Pearsons Engineering Services Signalling team have successfully delivered projects for Bombardier Transportation over the last 18 months including:

1. Crossrail - Installation of DOO CCTV Equipment at RIDC Melton
Installation of a DOO CCTV system simulating the layout that could exist at a 2 platform station. The configuration comprises 2 DOO wayside cabinets positioned on concrete plinths and 2 DOO Leaky feeder cables suspended on either side of the track on a metal framework (Unistrut) dug into the ground. This recreates the position that the cable would be mounted on the face of a platform. Also included is one representative camera stanchion (Crossrail prototype version) mounted beside the track positioned as it would be on a real platform.

2. Crossrail - Installation of AWS/TPWS Equipment at RIDC Melton
Installation of 5 x Train protection warning system (TPWS) as per the Thales hand book number 13764 this includes TSS & OSS loop aerials track side enclosures, line side cables and tail cables. Location of equipment Old Dalby & Saxelbye.

Installation of 5 x Automatic warning systems (AWS) as per the signalling installation hand book NR/GN/SIG/11210 this includes Permanent magnets and electro magnets and all associated sleeper mounting equipment. Location of equipment Old Dalby and Saxelbye. Installation of the two control panels at Saxelbye and Old Dalby test track

3. Crossrail - Installation of European Train Control System (ETCS)
Installation of ETCS balises and control equipment at Melton test track. This was to permit representative testing of the Class_345 train in ETCS Levels NTC (both AWS/TPWS and CBTC), Level 1 and Level 2, including dynamic transitions between levels.

Documentation ref: Installation Specification 2010-1.4 DAA

All lineside installation was installed in accordance to the signalling installation handbook ref: NR/L3/SIG/11303. Disconnections boxes and signage marker boards were mounted on post & skates and post mix was used to fix them in place.

The installation of power supply units and the installation of power and control cables throughout both sites.

We have also carried out installation in the REB's as per the client's design. This project has allow Bombardier to test their system and allow for any changed require for further installation. PES staff have attended ERTMS training with Signet solutions covering Level 1,2 & 3. Our staff have been working with the client programming the balise and LEU equipment of and on site. After programming had been completed we tested the ETCS equipment to insure the telegram had been down loaded correctly.

All of the above projects have been installed to the client's design and are for the testing of their equipment at Old Dalby test track. Our recent project has included the installation of the balise units over a 3 mile section of the track.

Successful testing on a test track doesn't mean that it will work in London.

It is very easy to be cynical about what is or isn't being done without full details being available.

 

[Tio Terry]

In a sensible project plan, the transitions between signalling systems (if you REALLY needed so many different ones) would have been identified at the start as a likely key issue, developed on a test track somewhere with all the changeovers, the software developed early in the project rather than at the last minute, and all run and tested so there was none of this early days but last minute prototyping and sudden surprises on the main project.

The issues described in this month's Modern Railways really sound like Train Software School 1.01.

To my certain knowledge the interfaces between signalling systems were known as high risk items way back in 2015. Yes, you REALLY do need as many and back then they also included ATP. This was overcome by the use of TPWS+, or enhanced TPWS depending on who is giving it a name, and ETCS in the Heathrow Tunnels. I retired two years later so lost track of who was doing what with the development of solutions.

One of the problems was that the project was being led by very competent Civil Engineers but that there was nobody strong enough at the top with a good Systems background who could champion such issues.

 

[kevin_roche]

The issues described in this month's Modern Railways really sound like Train Software School 1.01.

I should start by saying I've never worked on Train Signalling software. It does seem a bit like a 1-dimensional version of the 2-Dimensional Air traffic system I worked on in 1975.

You can often tell when a software development is going badly by the version numbers. On Crossrail we have seen a series of very different numbers. So far I recall seeing A03-003, PD-3, Y0.230, Z1.0, PD+8, Z0.100, PD+10, PD+11, and now we have TR2.

 

[Nicholas Lewis]

I'm told by a well-placed source that a lot of the increased cost and delay to Crossrail is due to the expensive consultancies not actually solving problems but simply creating more work for themselves in order to extract more money from whomever is dumb/naive enough to pay up. No change from normal there then! Especially in the current situation, where much of the consultancies' normal work has dried up.
Having said that, this source is known for his dislike of and contempt towards consultants, so there may be a bit of exaggeration going on!

This is par for the course. The industry has few leaders who have inner confidence anymore and use of consultants has become endemic to protect there backsides

 

[JonathanH]

I'm told by a well-placed source that a lot of the increased cost and delay to Crossrail is due to the expensive consultancies not actually solving problems but simply creating more work for themselves in order to extract more money from whomever is dumb/naive enough to pay up. No change from normal there then! Especially in the current situation, where much of the consultancies' normal work has dried up.
Having said that, this source is known for his dislike of and contempt towards consultants, so there may be a bit of exaggeration going on!

This is par for the course. The industry has few leaders who have inner confidence anymore and use of consultants has become endemic to protect there backsides

Is the implication of these comments that large infrastructure projects should solely be specified / built / developed by people directly employed by the organisation running the project? Is that in any way practical?

 

[Starmill]

This is par for the course. The industry has few leaders who have inner confidence anymore and use of consultants has become endemic to protect there backsides

'inner confidence' is of course maintaining in-house full time capability to do what your consultant delivers once in a blue moon, and leaving such personell resources either idle or grossly under utilised most of the rest of the time.

Competitive tenders which are written by experienced staff for requirements which are clear to management at the tendering authority and also all management at the potential bidders can provide value for money services - far better than if the tendering authority tried to hold all of the expertise itself for small but essential portions of projects. Ill-drafted briefs and less than transparent applications of scoring are the scourge of the modern competitive tender. Of course, this does not always happen. That's a discussion for another thread mainly.

 

[Meerkat]

So what you are saying is that the procuring body needs a permanent core of people who know what they are procuring and see through the consultancy/builder bull?

 

[Philip Phlopp]

So what you are saying is that the procuring body needs a permanent core of people who know what they are procuring and see through the consultancy/builder bull?

They need to know what to procure.

What is available, what is needed and what is procured should be the same thing. It often isn't.

 

[Meerkat]

They need to know what to procure.

What is available, what is needed and what is procured should be the same thing. It often isn't.

I am just speaking from experiencing consultants in finance. They could totally take the mick because they were being project managed either by permies without the knowledge to call them out, or by consultants acting as project manager, and the consultants don’t like to show each other up as they rely on each other for contacts and recommendations into their next jobs!

 

[Philip Phlopp]

I am just speaking from experiencing consultants in finance. They could totally take the mick because they were being project managed either by permies without the knowledge to call them out, or by consultants acting as project manager, and the consultants don’t like to show each other up as they rely on each other for contacts and recommendations into their next jobs!

The rail industry is rather different to the finance industry though - if consultants continue to get Crossrail wrong, it'll be evident in it not opening, or in ORR (or even RAIB, heaven forbid) having to deal with issues that arise. You finance guys, if you get something wrong and create a shed load of bad debt can do something with it, you can use to offset tax for example, we can't just keep hold of a signalling system that doesn't work, ignore it for a couple of years and then use it when it suits us.

 

[Meerkat]

The rail industry is rather different to the finance industry though - if consultants continue to get Crossrail wrong, it'll be evident in it not opening, or in ORR (or even RAIB, heaven forbid) having to deal with issues that arise. You finance guys, if you get something wrong and create a shed load of bad debt can do something with it, you can use to offset tax for example, we can't just keep hold of a signalling system that doesn't work, ignore it for a couple of years and then use it when it suits us.

Ha!
Actually most finance projects are either handovers (take overs, outsourcing, office closures) or software and system changes (so relevant to railway).
Failure can be covered by blaming the client, or the supplier, or the permies (sound familiar?), or you descope it, chuck in workarounds (they wont be operating them so not their problem), and leave at completion date so they can put ‘successfully completed’ on their CVs and ‘well it was fine when I left it’
I’m not bitter though, as you can probably tell!

 

[Roger B]

Ha!
Actually most finance projects are either handovers (take overs, outsourcing, office closures) or software and system changes (so relevant to railway).
Failure can be covered by blaming the client, or the supplier, or the permies (sound familiar?), or you descope it, chuck in workarounds (they wont be operating them so not their problem), and leave at completion date so they can put ‘successfully completed’ on their CVs and ‘well it was fine when I left it’
I’m not bitter though, as you can probably tell!

I worked in finance, and lost count of the projects managed by consultants that failed to deliver to time / cost / quality, usually disguising this by changing scope or attributing blame to poor specification, etc. And they'd bring one or two experienced people over, together with a host of newbies, who we were expected to train on the job, and who were being charged to us on eye-watering daily rates. No bitterness here, either!

 

[Belperpete]

Getting possessions on the Crossrail side to test the transition probably aren't impossible; after all they managed it with Heathrow tunnels which are at least as busy. That the stations in the Core aren't complete is probably the biggest blocker to the Core opening.

I agree that the stations STILL not being complete is likely to be a major blocker. On a railway where so much is integrated, having even one part not ready can impact on the rest. Things like tunnel ventilation, platform doors, emergency alarms now all impact on the signalling, so if one is not ready, it will impact.

I do not see why possession of the operational lines would be necessary to test the transitions, as they are not on what would then be the operational railway. Yes, the testing would need trains running on and off the operational railway, but these could surely be integrated with the service trains. From what I recall, if a train coming from Westbourne Park doesn't transition, it will be stopped by the TPWS well before it reaches what will then be operational Crossrail lines. Likewise if a train leaving Crossrail fails to transition, it will be stopped after the operational Crossrail lines. All they need to do now is to test that trains can be safely signalled on and off the link lines under CBTC signalling.

 

[Tio Terry]

I agree that the stations STILL not being complete is likely to be a major blocker. On a railway where so much is integrated, having even one part not ready can impact on the rest. Things like tunnel ventilation, platform doors, emergency alarms now all impact on the signalling, so if one is not ready, it will impact.

I do not see why possession of the operational lines would be necessary to test the transitions, as they are not on what would then be the operational railway. Yes, the testing would need trains running on and off the operational railway, but these could surely be integrated with the service trains. From what I recall, if a train coming from Westbourne Park doesn't transition, it will be stopped by the TPWS well before it reaches what will then be operational Crossrail lines. Likewise if a train leaving Crossrail fails to transition, it will be stopped after the operational Crossrail lines. All they need to do now is to test that trains can be safely signalled on and off the link lines under CBTC signalling.

Maybe the testing needs to test that the TPWS does actually stop the trains as it's supposed to? Also, of course, the reverse! Maybe that's why the testing needs possessions?

 

[Nicholas Lewis]

BR was efficient in what it did as it had sufficient in house resources to engineer and manage its projects. At privatisation most of that expertise ended up in the contractors and Railtrack was weakened from the outset. Furthermore it was lead by outsiders who then resorted to using consultants

Maybe the testing needs to test that the TPWS does actually stop the trains as it's supposed to? Also, of course, the reverse! Maybe that's why the testing needs possessions?

One would have thought that they've recreated the transition infrastructure at Old Dalby to prove the software and operation of the train under various signalling configurations and failure scenarios first so that live testing at Westbourne Park is proving it all works as designed rather than basic testing.

 

[Tio Terry]

BR was efficient in what it did as it had sufficient in house resources to engineer and manage its projects. At privatisation most of that expertise ended up in the contractors and Railtrack was weakened from the outset. Furthermore it was lead by outsiders who then resorted to using consultants

One would have thought that they've recreated the transition infrastructure at Old Dalby to prove the software and operation of the train under various signalling configurations and failure scenarios first so that live testing at Westbourne Park is proving it all works as designed rather than basic testing.

Yes, you can test the design off site but you can only test the installation on site, that may require possessions.

 

[Belperpete]

Maybe the testing needs to test that the TPWS does actually stop the trains as it's supposed to? Also, of course, the reverse! Maybe that's why the testing needs possessions?

Yes, but such testing would take minimal time, and should be amply covered in a single out of hours (eg night-time) possession.

 

[Class 170101]

One would have thought that they've recreated the transition infrastructure at Old Dalby to prove the software and operation of the train under various signalling configurations and failure scenarios first so that live testing at Westbourne Park is proving it all works as designed rather than basic testing.

It won't just be the operation of the train thats being tested but that the new signalling interfaces with the old signalling and new signalling such that a driver leaving (and entering) the Crossrail core will receive the correct sequence of signal aspects when transitioning from one signalling system to another and that secondly both adjoining systems understand each other at the junctions affected and a route set from and old route won't allow a route set from the tunnel. So for example a route set from Liverpool Street (High Level) to Stratford (platform 8) on the Electric Lines means that a route won't and can't be set from the crossrail tunnels to Stratford (Platform 8) at the same time equalling a 'wrong' side signal failure and a collision potential at Stratford.

 

[hwl]

Yes, but such testing would take minimal time, and should be amply covered in a single out of hours (eg night-time) possession.

@Class 170101 and @Tio Terry have a good understanding of the complexity of the situation. It is far more complex than you appear to think.
The basic testing was done pre covid and there have been train and signalling software iterations since then.
On the western side line speed transitions (the big problem) require a relief line block and certain points locked out as a westbound CR train can theoretically reach line 2 if TPWS doesn't work (AWS not working was recent issue). Multiple trains are also needed in a coordinated way to do testing. On the eastern side everything is now at multi train testing which needs a reasonable length of the electric lines blocked to stack the trains up.
The Shenfield - Paddington crossrail trains will also do driverless auto reverse in the turnback sidings between the running lines at the western transition location (with passenger over carry allowed).

== Doublepost prevention - post automatically merged: 31 Aug 2020 ==

BR was efficient in what it did as it had sufficient in house resources to engineer and manage its projects. At privatisation most of that expertise ended up in the contractors and Railtrack was weakened from the outset. Furthermore it was lead by outsiders who then resorted to using consultants

One would have thought that they've recreated the transition infrastructure at Old Dalby to prove the software and operation of the train under various signalling configurations and failure scenarios first so that live testing at Westbourne Park is proving it all works as designed rather than basic testing.

They have but lots of non basic testing needed.

 

[Meerkat]

The Shenfield - Paddington crossrail trains will also do driverless auto reverse in the turnback sidings between the running lines at the western transition location (with passenger over carry allowed).

That could be mildly alarming - miss your stop and then the driver walks past whilst the train is moving!

 

[hwl]

That could be mildly alarming - miss your stop and then the driver walks past whilst the train is moving!

They will try to clear at Paddington but can't always guarantee. The announcements should keep mentioning next stop Paddington till it reaches the platforms again.
A full walk-though train design is a pre requisite. (LU had some passengers fall between cars on out of service trains)

 

[Taunton]

That could be mildly alarming - miss your stop and then the driver walks past whilst the train is moving!

They will try to clear at Paddington but can't always guarantee. The announcements should keep mentioning next stop Paddington till it reaches the platforms again.
A full walk-though train design is a pre requisite. (LU had some passengers fall between cars on out of service trains)

In other words, what we have had here on the DLR for the last quarter-century. With no walk-through.

 

[Meerkat]

They will try to clear at Paddington but can't always guarantee. The announcements should keep mentioning next stop Paddington till it reaches the platforms again.
A full walk-though train design is a pre requisite. (LU had some passengers fall between cars on out of service trains)

Does the driver have to wait for the train to leave Paddington before starting to walk back? Will they have plenty of time?

 

[hwl]

Does the driver have to wait for the train to leave Paddington before starting to walk back? Will they have plenty of time?

Yes and Yes

 

[Meerkat]

Yes and Yes

Does the driver get a loo break at the other end of the line?

 

[hwl]

In other words, what we have had here on the DLR for the last quarter-century. With no walk-through.

Each DLR unit is walk through, the the trains of several units aren't overall. On the DLR there is no opportunity for a passenger to fall between vehicles as it is either walkthrough or end of unit. Think of any of the deep tube as comparison (e.g. Northern / Bakerloo etc.) for fall opportunities.

== Doublepost prevention - post automatically merged: 31 Aug 2020 ==

Does the driver get a loo break at the other end of the line?

There are certainly staff only toilet facilities at Paddington so I suspect some "stepping up" type arrangement could be used.

 

[The Ham]

So what you are saying is that the procuring body needs a permanent core of people who know what they are procuring and see through the consultancy/builder bull?

Part of the issue can be down to the fact that few people have an overview of everything that they have to deal with on a project.

I once worked on a project (highways, not rail, but as there's less "departments" to deal with it highlights how much of an issue is likely to be with something which is more complex) where one of the large multi disciplinary consultants was designing the scheme for the contractor.

Problem was that they designed the drainage to pass through the middle of a medium pressure gas main which they had also designed the protection of.

Fortunately, having had a good understanding of the project we were able to highlight this to the client fairly quickly and it was all sorted out before they got to site. Otherwise you can only imagine the costs as the scheme grinds to a halt whilst a work around is sorted out.

The reason that it happened was that drainage and the structure designs were done in different parts of the country and the checking was done in each department rather than by someone who knew the overall scheme.

It's the sort of error which is easy enough to do (which is why I won't say which company did it), however it should also be fairly easy to protect against.

Things like combined drawings (which look a right mess but highlight which bits need to be cross checked for clashes) are helpful as then even someone with zero understanding of what's being shown can ask for clarification as to if (say) that cable can cross that pipe at that location of the two departments who created the drawings showing those elements.

It takes time and therefore costs money, but it's small change compared to the costs which can happen if there's significant delays on site, especially if they then have knock effects for future parts of the project.

I don't know if such issues arose on Crossrail, so the above isn't suggesting that it did. Rather just highlighting how sometimes things can go wrong if there's not the right people asking for the right things.

 

[Taunton]

That's quite normal. CAD systems nowadays provide "clash detection" (as it is known) specifically for this. Of course, that does not trap all errors, such as something missed out completely.

Like you I work with multiple projects of different types, and it's not only clashes. A recent useful innovation is the ability to walk through/fly through images of the completed project, derived from CAD drawings, which catches things like signs which cannot be seen from certain positions, etc. All is dependent on everything being in one computer model, which still doesn't happen.