Big data and data anyalitics - does the rail industry make use of this?

[Carlisle]

What even the number of times a toilet gets flushed?

There’s probably already a fairly well established system on some stock that that’ll send notifications when water tanks are or nearly empty and waste tanks full, to save standing in the vestibule with a clipboard.

 

[takno]

Telefonica (o2) has a team that sells anonymous mobile phone data to transport and other sectors.

Ugh. How long will it take for somebody to reconstruct that back into personally identifiable information? Glad I got shot of them.

 

[HowardGWR]

Ugh. How long will it take for somebody to reconstruct that back into personally identifiable information? Glad I got shot of them.

Hopefully longer than it will take InfoBleep to correct his thread title - please IB?

 

[deltic]

Ugh. How long will it take for somebody to reconstruct that back into personally identifiable information? Glad I got shot of them.

All mobile phone companies sell the same data - it is now widely used in the transport sector. Mobile phone and related data is now widely used to track people in many areas such as shopping centres, airports, stadia etc

 

[takno]

All mobile phone companies sell the same data - it is now widely used in the transport sector. Mobile phone and related data is now widely used to track people in many areas such as shopping centres, airports, stadia etc

I know. I literally am waiting to see how long it will take somebody to reconstruct the data to significant PII so I can get popcorn and watch the whole thing turn into a scandal. On the face of it there's no reason why it should be particuarly difficult to do.

I'm mostly glad to be shot of O2 because I found their network terrible.

 

[Re 4/4]

Not only do the railways gather a lot of data, you can sign up to get access to some of the feeds - I presume this is what drives RTT, traksy and friends (though RTT seems to do quite a bit of processing of its own, and is interested if you can provide additional GPS data).

To me this sounds like a great area for someone interested in big data to do a project.

 

[takno]

Not only do the railways gather a lot of data, you can sign up to get access to some of the feeds - I presume this is what drives RTT, traksy and friends (though RTT seems to do quite a bit of processing of its own, and is interested if you can provide additional GPS data).

To me this sounds like a great area for someone interested in big data to do a project.

It is indeed where RTT and Traksy source their data, and given the format the data comes in we both do a fair amount of processing to get it into maps and readable schedules

Bear in mind the Network Rail feeds have at least a four month wait to get access to the data at the moment.

If you aren't looking for the signal-by-signal data then https://datafeeds.nationalrail.co.uk/#/registration is better formatted, and gives you access to data from some other sources. I believe it also has less of queue to access it. There are some restrictions on how you can display the data, which is why some sites stick with the Network Rail feeds instead.

 

[HH]

People are using big data. It's just that they're not all shouting about it - for various reasons.

 

[takno]

People are using big data. It's just that they're not all shouting about it - for various reasons.

Hardly anybody who is actually doing big data is shouting about it at all. Given the ability to run the processing on cheapish cloud hardware none of the data sets seem all that big anymore, and most of the analysis is just the same statistical techniques people have been using for years. Arguably it's become so cheap and pervasive that the quality of the analysis being done is lower than it has ever been, and the results less meaningful. The lustre starts to fade when you realise that you've spent 3 months and a hundred grand on compute only to prove what the subject domain experts already worked out with a well-chosen subset of data and an Excel spreadsheet.

 

[HH]

Hardly anybody who is actually doing big data is shouting about it at all. Given the ability to run the processing on cheapish cloud hardware none of the data sets seem all that big anymore, and most of the analysis is just the same statistical techniques people have been using for years. Arguably it's become so cheap and pervasive that the quality of the analysis being done is lower than it has ever been, and the results less meaningful. The lustre starts to fade when you realise that you've spent 3 months and a hundred grand on compute only to prove what the subject domain experts already worked out with a well-chosen subset of data and an Excel spreadsheet.

And yet spending a couple of grand can grant you valuable insights. I guess it all depends, like most things.

 

[alistairlees]

Hopefully longer than it will take InfoBleep to correct his thread title - please IB?

Seconded!

 

[Unixman]

The following is a slightly nerdy techie subject. Apologies if I've not explained anything in plain English. I did try to.

I went to a conference today on data and how it can be used for the public good. Everything from using data to better target help where it is needed most to transparency and privacy of data.

Do Network Rail and / or the TOCs / RDG use data and data mining to improve their services and processes? For example to workout where best to target resources or what issues are likely to lead to delays and how best to mitigate against them?

With regards to implementing DOO, what kind of data analysis would they have done on the introduction of such a thing? And would they have also studied the issues that might occur from its introduction. Note I don't want to discuss the merits of DOO, which I personally disagree with, just thd data analysis that may or may not have been done.

The conference also talked about how one can capture information and can make use of it. How is railway data captured? For exanole do drivers still fill out handwritten forms on why their trains were delayed or is it now a digital form?

The conference looked at the sharing of data and the combining of it to produce better knowledge and outcomes across multiple organisations, as well as trying to stop silos. Obviously it needs to respect GDPR but it is possible to do that.

Is data shared much in the rail industry?

Could artificial intelligence be use at all in the rail industry?

I find the way data and it's uses is heading, very interesting. I say data and not digital data, as even paper records can play a part.

Given that software like Hadoop allows you to search for and find the proverbial needle in a haystack and is essential for deep-mining of data, then I would expect that it is. Or at least should be.

 

[AlterEgo]

Surprised nobody has mentioned Traffic Management (TM) yet.

 

[matt_world2004]

The following is a slightly nerdy techie subject. Apologies if I've not explained anything in plain English. I did try to.

I went to a conference today on data and how it can be used for the public good. Everything from using data to better target help where it is needed most to transparency and privacy of data.

Do Network Rail and / or the TOCs / RDG use data and data mining to improve their services and processes? For example to workout where best to target resources or what issues are likely to lead to delays and how best to mitigate against them?

With regards to implementing DOO, what kind of data analysis would they have done on the introduction of such a thing? And would they have also studied the issues that might occur from its introduction. Note I don't want to discuss the merits of DOO, which I personally disagree with, just thd data analysis that may or may not have been done.

The conference also talked about how one can capture information and can make use of it. How is railway data captured? For exanole do drivers still fill out handwritten forms on why their trains were delayed or is it now a digital form?

The conference looked at the sharing of data and the combining of it to produce better knowledge and outcomes across multiple organisations, as well as trying to stop silos. Obviously it needs to respect GDPR but it is possible to do that.

Is data shared much in the rail industry?

Could artificial intelligence be use at all in the rail industry?

I find the way data and it's uses is heading, very interesting. I say data and not digital data, as even paper records can play a part.

Tfl Do extensively they can usually predict pretty accurately how many people are effected by a service change. What incidents would cause crowding levels to the point of leaving people behind.

 

[squizzler]

The railway industry was the first to adopt the electric telegraph, modern management techniques, and British Rail was one of the first businesses in this country to get a mainframe computer for business analytics and to digitise all their vehicles (with TOPS back in the 1960s). Railways have always been at the forefront of management techniques including use of data.

It's a welcome development to see (if Silicon Valley wibble is to be believed) minicab clearing houses (Uber, Lyft), takeaway dinner ordering agencies (Just Eat, Deliveroo) and mail order firms (Amazon) running their service with the degree of computerised asset management that Western railway administrations have enjoyed for the last few decades!

 

[infobleep]

Hopefully longer than it will take InfoBleep to correct his thread title - please IB?

What is wrong with it? The spelling? I corrected one word that might be wrong.

 

[krus_aragon]

What is wrong with it? The spelling? I corrected one word that might be wrong.

"anyalytics" vs "analytics"