They're unlikely to look at your Oyster Card use - but we can't be certain that they won't.
In any case, you've already used the Oyster Card, so there's nothing you can do about it right now. When the railway write to you, read the letter very carefully (and it could be a good idea to share it...