You can leave without cancelling and come back visa exempt or tourist visa, no problem, you're not breaking any rule.
But if one day you want to apply for a new NON-O visa, you will have a problem because you didn't cancel the current one. So you will need to do a backward cancel. My suggestion is cancel it now, it's not difficult.