If your birth times are accurate and you are the inner chart, then his Venus is right on your descendant. That is what you see when you see him; so there is a very powerful draw from him to you. Additionally, regardless of the birth times, your Venus and his Mars trine by sign. There are other connections as well.
However, synastry doesn't tell if or how someone will respond to another as that is dependent upon causes and conditions that are difficult to discern in their entirety in a birth chart, or anywhere, really. So knowing why you are so attracted to him doesn't tell you if you ought to act or not. A relational horary chart might be a better way to try to answer that question.