Yes, but getting a visa was never the main overhead involved in moving to Europe. That's why in practice very few British people ever used that ability outside of retirees going to Spain, and why it didn't play a part in the debate. Some people liked the idea that they could in theory live in France or Germany, but moving to the US, Australia or Canada has always more popular for British workers despite being harder.

I'm one of the few Brits that actually did move to Europe, specifically to Switzerland at a time when being in the EU didn't help, so I went through the usual immigration process. The paperwork wasn't an issue. 99% of the work was language learning and social integration. Hence why Anglophone countries get the bulk of the UK emigration.