The government failed to tackle immigration - arguably they had one job in this Parliament given to them by voters - and are paying very dearly. It wasn’t for the want of ideology or a political manoeuvre either, but I don’t think you can look at this, see a ton of voters going to Farage’s party, and conclude “well the Tories moved too far right”.
The problem is that their words don't match their actions.
A percentage of people thought that Brexit would result in fewer people coming here (it may not be a big percentage but it's also not zero), they are now mostly upset that net migration is higher.
However it's no surprise that it is, it's been shown that if you make it harder for people to move they'll settle where they "need" to be. Whilst not as easy to move back and forwards between the EU and the UK as between Mexico and the US, when experience from the US e that by making the boarder harder to cross they stop crossing. Only they do so by crossing the boarder into the US and then staying there.
Whilst the numbers coming here from the EU are down, those doing so are generally staying rather than coming for a bit and then going back and possibly even then maybe coming again for a bit and then going back.
For instance by making it hard to find here to do seasonal work, people either have to come here or not. If they aren't coming from the EU, then they'll be coming from elsewhere. Add to that making it harder for people to move abroad, then the numbers leaving could well be down.
The outcome is that migration is up.
Unless the government were to limit migration (and the impact that could have on the economy) then this was a likely outcome of Brexit.
In government they possibly understood this, but wanted to look like they were doing something, so take a lot about small boats, which was never going to change the numbers by much but took up a lot of time and coverage so looks like they were doing something.
Thing is that will only let you get away with it for a short time before people start to see what you're doing and shift away from that thinking or demand someone else to do what they want (of course given age is a factor on people's thinking time may well alert the overall perception - especially given that getting older doesn't have such a strong link to becoming more right wing at it once did).