Oyster card asynchronicity

[John Palmer]

Moderator note: Split from

Thameslink Notice of intention to prosecute

Hi, I'm in a bit of a state, I've just received a letter from Thameslink informing me that they intend to prosecute. I have drafted a response, and would love some eyeballs on it from someone with some experience. It feels insanely heavy handed to do this, especially as it was all down to my...

www.railforums.co.uk

Apologies if I am betraying my ignorance of the way in which the Oyster system works, but I remain puzzled by the fact that a credit balance was still being attributed to the card when the OP touched in at Ferme Park Road. Presumably that credit balance was also returned to the Finsbury Park reader when the card was touched in there too, indicating that there was sufficient credit to cover the fare to the next station with the consequence that the card was accepted for travel per condition 3.4 of Oyster Conditions of Use on National Rail Services.

I can understand why latency effects upon the FUL facility could have postponed the moment at which readers would have recognised the card's cancellation – in this case when the exit at Kings Cross was attempted. But that doesn't explain why a credit balance was still being attributed to the card some nine months after that balance was refunded to the cardholder, assuming the refund request was honoured in full. In terms of the system's design, what possible reason could there be for postponing an update to the record of the balance credited to the card for a period of many months after that balance has been reduced to zero by a refund to the cardholder? If, instead, the record of the card's credit balance was updated as soon as the refund was made, a subsequent attempt to touch in would fail because the system would show insufficient funds being available to meet the minimum fare payable.

(Edit) Actually, I suspect I have misunderstood the system's design, as it seems that one record of the current balance is held upon the card and it is this rather than a centralised record that gets interrogated by the reader. Is this correct? The fact remains that the system's design seems to be such as to preserve the card's validity for an indeterminate period until its cancellation has fully propagated.

 

[swt_passenger]

[…]
But that doesn't explain why a credit balance was still being attributed to the card some nine months after that balance was refunded to the cardholder, assuming the refund request was honoured in full. In terms of the system's design, what possible reason could there be for postponing an update to the record of the balance credited to the card for a period of many months after that balance has been reduced to zero by a refund to the cardholder? If, instead, the record of the card's credit balance was updated as soon as the refund was made, a subsequent attempt to touch in would fail because the system would show insufficient funds being available to meet the minimum fare payable.

The PAYG credit balance is effectively stored as data on the card, and after the refund dealt with on the phone, the card data was effectively stale and no longer in sync with the copy of its data held in the back office system. Until the card was presented to the system I believe this stale data would remain on the card in perpetuity.

Clearly the call handler cannot alter the data on the physical card at the time of the conversation. But with hindsight surely they should have briefed the card user about this latency. “I have refunded your balance but your card will not catch up with this until you next attempt to touch in.”

 

[island]

Apologies if I am betraying my ignorance of the way in which the Oyster system works, but I remain puzzled by the fact that a credit balance was still being attributed to the card when the OP touched in at Ferme Park Road. Presumably that credit balance was also returned to the Finsbury Park reader when the card was touched in there too, indicating that there was sufficient credit to cover the fare to the next station with the consequence that the card was accepted for travel per condition 3.4 of Oyster Conditions of Use on National Rail Services.

I can understand why latency effects upon the FUL facility could have postponed the moment at which readers would have recognised the card's cancellation – in this case when the exit at Kings Cross was attempted. But that doesn't explain why a credit balance was still being attributed to the card some nine months after that balance was refunded to the cardholder, assuming the refund request was honoured in full. In terms of the system's design, what possible reason could there be for postponing an update to the record of the balance credited to the card for a period of many months after that balance has been reduced to zero by a refund to the cardholder? If, instead, the record of the card's credit balance was updated as soon as the refund was made, a subsequent attempt to touch in would fail because the system would show insufficient funds being available to meet the minimum fare payable.

(Edit) Actually, I suspect I have misunderstood the system's design, as it seems that one record of the current balance is held upon the card and it is this rather than a centralised record that gets interrogated by the reader. Is this correct? The fact remains that the system's design seems to be such as to preserve the card's validity for an indeterminate period until its cancellation has fully propagated.

In order to meet the need for Oyster cards to open ticket barriers in an acceptable timeframe (I believe 0.3 seconds is the maximum), the balance on a card, along with other status information such as discount entitlements, is stored locally to the card. What you describe as "latency effects upon the FUL facility" applies as much to the card's balance as to its cancellation status.

Bear in mind the underlying technology is approaching 20 years old.

I have asked the moderators to split this side discussion to a new thread as it does not advance the goal of assisting the OP with defending the criminal offence they find themselves charged with.

== Doublepost prevention - post automatically merged: 21 Jan 2022 ==

Clearly the call handler cannot alter the data on the physical card at the time of the conversation. But with hindsight surely they should have briefed the card user about this latency. “I have refunded your balance but your card will not catch up with this until you next attempt to touch in.”

The script to be followed is much more simple and goes "please dispose of the card and do not attempt to use it again".

 

[John Palmer]

Thanks for confirming what I had belatedly discovered to be the basis on which the Oyster system works.

Oyster's system design involves an implicit assumption that there will be cases in which a card will continue to function as a valid card for travel acceptance purposes until it receives the instruction that the data it holds is to be modified so as to show that it has been cancelled. It seems that, due to latency effects in the system, such an instruction may not be implemented by a change to the data on the card for an indeterminate period after that instruction is triggered by interrogation of the FUL background list.

For some reason TfL treat a request for credit upon an Oyster card to be refunded as one that must lead to its cancellation. Apparently this is not what the OP desired in the thread from which this one has been split, as I understood him to say that he was seeking a refund of the credit upon the card but not its cancellation. Where, as in this case, such a credit was available for refund, TfL could prevent the card's re-use immediately by making refund of the credit conditional upon the card's physical surrender. If not so surrendered, such credit as remained upon it would permit the card's continued use in the usual way until the balance was reduced below the threshold for travel acceptance. But instead, TfL is evidently content in such circumstances to permit such a card to remain in the customer's hands in the knowledge that it may continue to be accepted for travel until such time as it receives the cancellation instruction.

I can't see any good reason for TfL to operate the Oyster system in a way that allows spurious credit balances to remain on the card after they have been refunded to the customer when it is so easy to prevent such a situation from arising. Nor do I see an obvious reason for treating a request for a refund of credit to be treated as a request for the card's cancellation. Indeed, such a practice seems inconsistent with the wording of Condition 5 of the Oyster Conditions of Use on National Rail Services. Condition 5.2 says “If you have pay as you go credit on your Oyster card but no longer need it, you can get a refund of the balance on your Oyster card at London Underground stations.” Condition 5.4 says “If you no longer need your Oyster card, you can get any remaining pay as you go credit and any deposit paid back by handing it in at any London Underground ticket office or by calling TfL Customer Services.” If a refund of credit is always going to lead to the card's cancellation, Condition 5.2 is redundant and the Conditions should instead make it clear that a request for a refund of credit will be treated as a request for the card's cancellation.

There is room for doubt whether the OP in the original thread was told that refund of his credit would involve cancellation of the card, for, if he had been told that this was the case he would presumably have told TfL that he was only seeking a return of his credit and not the cancellation of the card – an outcome he might well have expected to achieve in view of the wording of Condition 5. Since the Conditions do not say, in terms, that a request for a credit refund will be treated as a request to cancel the card, I am led to wonder whether TfL's practice of doing so is permitted by the terms of its contract with the cusmtomer as defined by the Conditions of Use.

 

[zero]

Is it not possible to hotlist an oyster (like can be done with a contactless device) so that it would show as invalid from the next touch?

If this can be done, once the card is cancelled from the back office it's irrelevant what the card thinks its balance is

 

[skyhigh]

Is it not possible to hotlist an oyster (like can be done with a contactless device) so that it would show as invalid from the next touch?

If this can be done, once the card is cancelled from the back office it's irrelevant what the card thinks its balance is

This is what happens

Now, apologies for not seeing this earlier, but the gap between cancellation and trying to use it again makes me suspect you may have stumbled upon an unintended consequence of TfL's system. When a card is cancelled remotely a record is set up to kill the card when it is next touched. This will use the faster universal load facility. Records only stay on FUL for a maximum of six months before being added to a background list. The background list is only consulted when a touch is received by the central system. Thus the bus journey will have been accepted and that will have triggered the transfer of the kill record to the active FUL list. That can take up to half an hour to be propagated to all gates etc, so it is possible that the touch in at Finsbury Park would also have worked. But by the time you came to touch out at Kings Cross the bullet will have been fired.

The issue with the OP in the original thread was that they didn't use the Oyster card until after the 6 months had elapsed

 

[infobleep]

I don't know if this is the same kind of issue but
last July my debit card changed to a new one and at some point, my auto top-up failed. Now TfL would have sent me an e-mail but I don't regularly check commercial e-mails sent to me and missed it.

So when I went to tap in recently it didn't work as my Oyster card was deactivated. They do this after a set number of days I was told. I think I was told a week.

I went online and saw the issue and pay off the remaining balance but the card still doesn't work. I couldn't see anything online about the card no longer working once I'd paid the outstanding balance but a member of staff did say I'd need to ring the helpdesk.

This I did and they refunded the balance on my card and I got a new one from the machine, complete with a discount added on.

I nos have a calendar reminder for 2025 to change my card details online.

I'm sure this has occurred in the past but I'd be travelling often enough to deal with it, within the time frame. Now I'm not.

 

[John Palmer]

infobleep, in your case it appears that the top-up failure occurred less than six months before your next attempt to touch in, and for this reason what MikeWh refers to as the 'kill record' was still available for immediate transmission to the validator via the FUL facility, and had yet to be transferred to the background list. The latency issue that's been identified only manifests itself when the instruction to de-activate the card has been transferred to that background list. A check against that list is triggered by touching in the card, at which point it appears that the pending deactivation instruction gets transferred back into the 'live' FUL facility, after which it again becomes available for immediate transmission to the next validator to which the card is presented. It seems, however, that this re-transfer to FUL is not instantaneous, and up to 30 minutes may elapse before it is acomplished.

I assume that in your case the balance on your Oyster card had become a deficit as a consequence of the top-up failure. In that case, even if the 'kill' instruction for the card had been transferred to the background list due to the passage of time since the last touch in, and was consequently not immediately available to the validator at your recent touch in, the card would not have been accepted for travel because it would have disclosed insufficient credit to cover the cost of the minimum fare from that station.

 

[infobleep]

infobleep, in your case it appears that the top-up failure occurred less than six months before your next attempt to touch in, and for this reason what MikeWh refers to as the 'kill record' was still available for immediinate transmission to the validator via the FUL facility, and had yet to be transferred to the background list. The latency issue that's been identified only manifests itself when the instruction to de-activate the card has been transferred to that background list. A check against that list is triggered by touching in the card, at which point it appears that the pending deactivation instruction gets transferred back into the 'live' FUL facility, after which it again becomes available for immediate transmission to the next validator to which the card is presented. It seems, however, that this re-transfer to FUL is not instantaneous, and up to 30 minutes may elapse before it is acomplished.

I assume that in your case the balance on your Oyster card had become a deficit as a consequence of the top-up failure. In that case, even if the 'kill' instruction for the card had been transferred to the background list due to the passage of time since the last touch in, and was consequently not immediately available to the validator at your recent touch in, the card would not have been accepted for travel because it would have disclosed insufficient credit to cover the cost of the minimum fare from that station.

Interesting. That makes sense.

 

[John Palmer]

The thread from which this was split has now been locked. Whilst I am aware that it had become subject to repetition of previously expressed views, it did seem to me that the case raised wider issues that might be worth further exploration.

In the other thread, it was apparent that the only offence with which the OP there was threatened was that of failing to hand over a ticket for inspection and verification of validity when asked to do so. I inferred from this the possibility that GTR regard a cancelled Oyster card as not being a ticket within the meaning of the relevant bylaw. Whilst I struggle to see a basis on which GTR might sustain that argument, I would be interested to hear why anybody might think otherwise, particularly in the case of a distributed processing system such as Oyster where, in certain circumstances, a card will continue to be treated as valid for the purposes of acceptance for travel until a cancellation instruction has propagated from the system's central data store.

I should also be interested to learn whether, when an Oyster card is marked as cancelled on the system's central data store (as distinct from the data on the card itself), the central record of card transactions and the current credit balance ceases to be accessible to the cardholder. A reading of the other thread raised that as a possibility and led me to wonder whether, in such a case, any means were available to the card holder for checking the card's status and balance other than interrogation of the card itself. If that is the case then the card holder is left with no practical means for discovering that the card has been cancelled until alerted to this by a touch in after the 'kill record' has propagated throughout the system – by which time it seems he may face prosecution for an offence.

 

[island]

In the other thread, it was apparent that the only offence with which the OP there was threatened was that of failing to hand over a ticket for inspection and verification of validity when asked to do so. I inferred from this the possibility that GTR regard a cancelled Oyster card as not being a ticket within the meaning of the relevant bylaw.

I on the other hand inferred the possibility that GTR have read the word "valid" into byelaw 18 (2) when it does not appear.

 

[John Palmer]

I on the other hand inferred the possibility that GTR have read the word "valid" into byelaw 18 (2) when it does not appear.

That is indeed another possible inference, though I would struggle equally hard to see how such an interpretation could be justified!