If you're expecting to go to any country and encounter only locals and local culture, you will be disppointed.
But otherwise, if you're not expecting a "pure unadultered cultural experience" where you will only encounter native Greeks in Greece, then who cares who lives where?
The Greek islands are beautiful nonetheless and there is still culture to be found. The local food and sightseeing is still in tact.