genetics...epi-genetics...hormones....they develop separately from other parts of our bodies...sexual "Identity" occurs in the brain...in utero and in neo-natal environments...
These studies have been going on for decades...they involve scientists doing objective studies that are subject to peer review for accuracy and logic.
Sorry, but the world isn't a simple as you'd like it to be.