Physical appearance can determine a lot.
The ascendant is also the most apparent of someone's personality.
So if someone expresses a lot of traits of a certain sign, chances are, it's their ascendant and not even their sun sign.
The ascendant also determines where planets fall in the houses, so finding an ascendant that makes sense for how your planets fall in your houses can also help to determine your ascendant.
It's actually pretty easy to figure out someone's ascendant with these techniques.