There are a million+ pure-blooded Khmer people on the Thai side, and numerous Khmer temples and ruins dotted all over a few provinces in lower Isaan. Is your claim that those should "remain on the Cambodian side" purely by virtue of being historically Khmer culturally? Whether you like it or not, Khmer culture is Thai heritage also.
While we're at it, there are Khmer-speaking people, and temples/ruins, in both Laos and Vietnam. As a Khmer ethno-state, does Cambodia have a claim on those too, in your view?
Being "built by Khmer kings" is not an appropriate criteria to assess this situation.