I'm very late to this party, so you are probably already sorted. I renewed my passport by mail in November 2021. I just followed the directions on the Embassy website. The only thing I messed up on was including something to do with sending me the new passport in Thailand (was an envelop or money? I don't remember), but they sent me an email (or they called) and I took care of it straight away. I thought it was quite easy.
I've never found photo shops to know the correct size. I've used 2 different shops in the last 15 yrs. I check the size needed for the forms (US passport, visa application, extensions, etc.) and tell them what I want. As someone else stated, you can trim the size that you currently have rather than going to another shop. (I remember many years ago one of our passports (my son or I) had a rectangular photo and they accepted it because the face size was correct).