You may get envious stares by other lonely men, but you'll also get stares of disgust from many who know the only reason she's with you. And if you show up to family functions with a girl who is younger than your own kids, the family will think you're a pervert.
So there is that. If you don't care what people think then you have nothing to worry about. The younger she is the more compounded it will be.
At some point too, she might get bored and leave you for a younger man deciding that all the money in the world isn't worth staying with an old fart with one foot in the grave.. And then you find yourself stuck at 75 taking care of the 2 year old you fathered. I've seen it happen.