That makes sense. I guess that goes along with the single life?
Well for some people it's much worse. Some people don't even get potential suitors.
I also think something about you personally contributes to this. People in this thread have already mentioned the pluto near your ascendant that may cause you to have an intimidating air. Traditionally, your Scorpio rising ruler Mars is in Aries, which makes it dignified.
This means that you might be regarded as Martial person - energetic, fighter, "sharp" and yes even intimidating. Interestingly, it is placed in the 5th house by quadrant -- which inclines you to express yourself mostly in the sphere of enjoyments and pleasures - one such area being romance. It is 6th by whole sign which actually means that it's in its "joy". All of these factors plus the fact that this is a nocturnal birth makes Mars a positive influence in the chart.
What puts the damper on Mars is Saturn. Saturn is the Malefic out of sect which increases his potential to do harm in the chart. He is in his domicile - Capricorn - and in a dominating square towards Mars - you. I take this to mean that what ever Saturn signifies oppresses and stops you in your tracks - and is quite successful at it. It thwarts (Saturn) your actions (Mars). And where is Mars located again? In the 5th house. Taking everything together, Saturn stops any actions that occur within the sphere of romances and pleasures.
Saturn, ruling both the 3rd and 4th -- signifies your family. Has it been the case that your family members stops you from pursuing relationships? Do they get in your way generally speaking?