Our economic system requires that we have a large working population to support those who aren't working (for whatever reason). Immigration of working-age people works out very well for us overall because those people come to country ready to work "out of the box" (i.e. they don't need to spend 18+ years being raised and costing the taxpayer money). My understanding of anti-immigration views is that there's plenty of unemployed people in this country who could work instead, so why should we import people? Realistically, I don't think that you could find a lot of like-for-like replacements for many jobs where particular qualifications, experience and work ethic are needed. The fact this situation exists is not pleasant, and has probably been caused in part by no plan to transition those who once did mining and metal bashing into more modern jobs. Nonetheless it seems to be the way things have gone and we can only move forward.
I think that if we wanted to go "fortress Britain", we'd have to make some changes to how see employment. If we want to keep up our current economic model, then we would have to inject a huge amount of money and resources into getting people off the dole, into employment and doing work. I don't have any idea how much this would cost, so I cannot say if it's a realistic option, although I'm sure it would be a vote winner in areas suffering from high unemployment. Alternatively, we accept that technology is removing the need for employment in the traditional sense, and instead push for automation and find new things for people to do. Provided everyone can be fed and housed this isn't necessarily impossible, although it would probably require a far more socialist culture than ours in order to work well. In any case, I don't think we could just slam the doors shut and pretend everything will be okay and as it was before.
My view is that if the population vote "in", then not much will change. There will be no political will to get people off the dole and into work whilst importing workers is cheaper (at least in the short term). If the population vote "out", not much will change either, as our way of life is dependent on trade agreements and immigration. We might see slightly reduced immigration, but I suspect that a lot will still be permitted in order to support our economic system.