My point is you can change anying, but it is what in your brain determines and makes you feel who you are. You can have body parts other than brain from everywhere, that will not change your identity.
The confusion you have is: you have a lot of non-self part, and you are now confused who you are, because you are assuming non-brain part can also determine or change identity.
There is subjective and perceived identity. The perceived is what you are seen as. Eg. A face change may make you look someone else, but you would never accept that you are now the person from whom you got your face is. This does cause a lot dilema, some are ethical, to the people who know you. there was a case of penile transplant in China. He eventually had it removed because his wife would not accept it. (If I remember correctly).