National Passenger Satisfaction Survey - Implicit bias?

[IKBrunel]

Passenger focus are conducting the National Rail Passenger Satisfaction Survey (NPS) this week. I've been given the form to complete at a station platform several times over recent years. What I've noticed is that this is always in the morning, before I catch a train on my way to work. I've never been given a form in the evening on my way home.

It seems to me that if passenger focus always give out forms in the morning, then this doesn't give an accurate picture of customer satisfaction. Compared to a service on the same route in the afternoon/evening, a morning train is less likely to:
Be delayed (due to knock-on chronology of problems).
Be dirty or littered (after overnight cleaning),
Have toilets out of operation (due to being blocked, out of water, etc).

I'm not trying to get these problems out of proportion or talk down the industry, I just think if the NPS is to be useful it needs to paint a reasonably accurate picture of the situation within resourcing constraints.
Just wondering what other people think?
for general info & previous results see http://www.passengerfocus.org.uk/research/national-passenger-survey-introduction

 

[SS4]

Additionally by giving them out in the mornings they are aiming for the peak time commuter crowd rather than leisure travellers

 

[Tetchytyke]

They're generally a bit more varied, it must depend on the rota for the survey staff. Last time I was given the NPS to fill in was on the Heysham Port train, so they don't just stick to the obvious places.

The NPS is flawed as they only deal with your most recent journey, rather than your overall experience.

 

[gordonthemoron]

LM morning trains out of Euston are often dirty and have toilets locked out of use because they haven't been cleaned

 

[Muzer]

The NPS is flawed as they only deal with your most recent journey, rather than your overall experience.

Overall experience would be much worse, since you are more likely to remember the times when things go wrong than the uneventful times, so these are likely to over-bias you towards saying the service is poor.


I think asking a variety of passenger types over a variety of times of day over a variety of days would simply be the best way. Even better, categorise these so you can get separate stats on (mostly) commuter, (mostly) regional and (mostly) long-distance/intercity journeys.

 

[Greenback]

I have been given a form in previous years on my way to work in the morning, and in other years on my way back in the afternoon. It seems pretty random to me. I also had one in the middle of the day on a leisure trip one year.

 

[asylumxl]

LM morning trains out of Euston are often dirty and have toilets locked out of use because they haven't been cleaned

Beats no train at all though surely?

 

[alex17595]

I got one on the hope valley line on an evening train to Sheffield.

 

[CC 72100]

Additionally by giving them out in the mornings they are aiming for the peak time commuter crowd rather than leisure travellers

I was given one on a Saturday lunch time last year - on a journey from Exeter Central to Exeter St Davids.

 

[londiscape]

Don't know about anyone else, but I would personally be inclined to complete such a survey in a LESS favourable manner if handed one in the morning on my way to work, just because I'm naturally bitchy and miserable at that time, not being a 'morning person'!

Be much more forgiving in the evening, however!

 

[maniacmartin]

I got one last year in the evening. I found the questions to be too leading and too similar - none of the multiple choice questions/answers covered my main reasons for dissatisfaction on the rail network.

 

[Hyphen]

I got given one of these a couple of years ago whilst standing at Sheffield waiting for the 0811 TPE to Cleethorpes.

I really didn't like the way the questions were laid out being very specific to this journey, partly because it pained me to give FirstGroup any praise at all (this feeling was mostly harboured having used an awful First S.Yorks bus to the station in the first place) and partly because that was the first time in memory that the 0811 actually left on time - pretty much every time I've caught that train it's been 4-10 minutes late leaving. That said, the only First TOCs I use regularly (FGW and FTPE) I do tend to rate well in my mind anyway, and my sole experience on the Caledonian Sleeper was great too.

 

[Aldaniti]

The NPS stats are fairly useless for a whole host of reasons, the only benefit is that it creates some work for a few people. The TOC's take them seriously - I wonder why!!! I was once asked to rate the toilet facilities on Merseyrail Electrics! Say no more.

 

[tsr]

I've had similarly stupid questions on surveys operated on behalf of specific TOCs, though. I've taken a seat on an FGW Pacer and then handed a form asking me what I think of onboard services such as the catering facilities.

 

[JohnGeddes]

There's another very fundamental bias - in that it only asks people who have chosen to travel by train. So they don't get to survey those who have rejected the rail option because fares are excessive, because timings are inconvenient, people for whom lack of reliable toilets are a deal-breaker, people who find timetables and fare restrictions too complicated - the list is endless.

To get a proper idea of what people feel about rail travel, you also need to survey those who DON'T travel by rail, to find out why they don't.

 

[TEW]

I've had similarly stupid questions on surveys operated on behalf of specific TOCs, though. I've taken a seat on an FGW Pacer and then handed a form asking me what I think of onboard services such as the catering facilities.

To be fair, that is a standard survey for all of FGW. Whilst it's been a couple years since I've been given one from what I remember it's a fairly comprehensive survey in general. Certainly seems to take a while to fill out at least.

 

[Clip]

There's another very fundamental bias - in that it only asks people who have chosen to travel by train. So they don't get to survey those who have rejected the rail option because fares are excessive, because timings are inconvenient, people for whom lack of reliable toilets are a deal-breaker, people who find timetables and fare restrictions too complicated - the list is endless.

To get a proper idea of what people feel about rail travel, you also need to survey those who DON'T travel by rail, to find out why they don't.

Im sorry but what??????


I know its early but you shirley can not be serious?

 

[cuccir]

To get a proper idea of what people feel about rail travel, you also need to survey those who DON'T travel by rail, to find out why they don't.

You're asking a different question to the one which the passenger survey is asking. The passenger survey is wants to know the experience of those who travel; it does not claim to speak for people who do not use trains.

 

[Gwenllian2001]

You're asking a different question to the one which the passenger survey is asking. The passenger survey is wants to know the experience of those who travel; it does not claim to speak for people who do not use trains.

But what about those people who have tried the train and have found the experience not to their liking?

Why do they not use the train? There could be many valid reasons, such as living too far from a station or that the station is inconvenient for their destination. On the other hand they might have had bad experiences travelling by rail; e.g. lack of seating, inadequate toilet facilities, unreliable services, cancellations at short notice and so on. There are any number of reasons why people do not use the train.

Simply seeking the opinions of regular rail users is a bit like preaching to the converted.

 

[cuccir]

But what about those people who have tried the train and have found the experience not to their liking?

Why do they not use the train? There could be many valid reasons, such as living too far from a station or that the station is inconvenient for their destination. On the other hand they might have had bad experiences travelling by rail; e.g. lack of seating, inadequate toilet facilities, unreliable services, cancellations at short notice and so on. There are any number of reasons why people do not use the train.

Simply seeking the opinions of regular rail users is a bit like preaching to the converted.

Those are interesting questions. But they are different questions. I agree that if they are not being asked by TOCs/DfT/Passenger Focus, then they are not doing their duty. That doesn't reduce the legitimacy of carrying out a survey of those who are using the network, however. As a social researcher, I can tell you that trying to do both in the same piece of research would be foolish (unless you have unlimited resources and can operate two separate research teams).

If a sufficient number of questionnaires are handed out as part of the National Passenger Satisfaction Survey, then a representative proportion of respondents will be infrequent rail users. I presume that the survey is designed to do this - if it isn't (which was the claim made at the start of this thread, but seemingly contradicted by the experiences of others), then I'd agree that the survey design is flawed.

 

[gordonthemoron]

i seldom travel by bus due to the lack of toilet facilities

 

[kieron]

Passenger focus are conducting the National Rail Passenger Satisfaction Survey (NPS) this week. I've been given the form to complete at a station platform several times over recent years. What I've noticed is that this is always in the morning, before I catch a train on my way to work. I've never been given a form in the evening on my way home.

They've got a description of how they conduct the survey here. That says about 2/3 of the people in their first few surveys were on an outbound journey, but that they have since altered their work pattern so the Spring 2013 survey had 49% of those questioned being on a return journey.

Perhaps there's something in there which shows if you were just unlucky not to see them on your way home.

I haven't read through it fully, so I don't know how representative a sample they use of the people who travel by train with each TOC at those times of year between 6am and 10pm.