Because we'd be trying to rejoin, Sweden is already in. Previous membership would not make a difference. The EU has always wanted all members to use the Euro since its inception and I think that is a condition of any new memberships now.
I do agree that the main downside to leaving the EU would be economic - always have - but in the long-term I still think it's worth it. That is, of course, an opinion.