As a dual citizen, how do I change my Non OA visa in my US passport to my Belize one?

you don't one suspects you will have to reapply under your Belize PP if that's what you want

Non-OA visas are only issued in your home country.

You will have to fly to Belize and apply for an OA visa there.

You cannot have your visa transferred from one passport to another. You will have to use it in your US passport until it expires.