So being with your daughter is found by others to be weird? That is their problem and yes it’s a problem.
I go a lot of places with my grand daughter and many people stare at us wondering what our relationship is. So we got tshirts in Thai and English saying grand father and grand daughter, no boom boom.