There is a Mattayom year 3 student in my class. A somewhat shy boy when he joined the school in M1, he became firm friends with another boy, far more self-confident than him. He seemed a lot happier and became more outgoing, but then he started his own Youtube channel about computer games which drew a lot of abuse online. He then retreated into his shell as it were, refusing to talk to anyone.

i asked his friend about him and he replied: เขาสูญเสียตัวตน  - he's lost himself  That doesn't sound quite right to my ears but i can't think of anything better.

Does anyone have any ideas?

How about, "He lost confidence in himself." "He became withdrawn."

From Longdo:

ตอนนี้เราต้องลากเขาออกมาจากที่นั่น ก่อนที่เขาจะสูญเสียตัวตนอย่างแท้จริง
Longdo translation: "Now, we've got to get him out of there before he loses himself completely."

It is always comforting to find a familiar English expression even when you have seen and described the situation. I told the story to some friends their reaction showed that they thought it very bad. The one better English speaker had that comforting expression which perhaps you are seeking; “loss of self esteem”.  I looked up self esteem  การประเมินค่าตนเอง so I find “he feels worthless” more acceptable in this case.  
I also agree with David, “self confidence” fits what you described. 

ตัวตน in this context covers things like identity, sense of self (worth)