Hi Nino, I've also thought about this problem in the past. I think an interesting and related question we could ask is, "Why are toilets separated by sex in the first place?"
Interestingly, many public spaces don't do this. Airplanes, for instance, don't have separate-sex toilets. Sometimes restaurants don't do this as well. Why is this so?
Something deeper that we could consider - and I think you've already done this - is how society arranges itself to perceive gender. If the popular consensus is that gender is binary, then of course people should use the restroom assigned to their sex.
But I think the first question is more relevant to the usage of public space. What sociological or political significance is the restroom? And why are they assigned specific sexes?