As I have a new US passport, am I required to go to immigration and transfer the visa to the new passport? Or is it ok, if I just leave and come back with my new passport? I am not interested in keeping the non-immigrant B visa that I have and I know I will get 90 days on arrival when I return from Malaysia, but do I really have to bother with getting the visa transferred to the new passport.

I had to go through this recently and yes, you have to take both passports to immigration, fill out a form, pay the fee and have the visa stamps transferred to the new passport... Don't ask me for specific details as I had my visa agent do it for me... 

Moving to the Visa forum for knowledgeable answers. 

Visas are not transferred. Only entry,  extensions of stay stamps, re-entry permits and etc issued by immigration are transferred to a new passport.

If you have a valid multiple entry non-b visa in your old passport you would use both passports on entry to use the visa in the old passport.