Actually, no, SS is not affected or garnished. There is literally no downside to not paying the loan back except not getting a new passport. But there SHOULD be -- it should be taken from SS, or VA benefits, or assessed at tax time. No reason the taxpayer should fund irresponsible people more than they already do.