Horary Charts?

I'm wondering something. A Horary Chart is drawn based around a question using the time, date, and place of where and when the question was asked. This is all relative to the person asking. But I'm curious, seeing it is relative to the individual asking how is it not a chart of where and when this person is in time and location? Literally?

How does the chart itself pertain to the question at hand?

What makes it an answer?

To note I am not a skeptic of Astrology. I've been studying natal, event, synnastry, and composite charts as a hobby since 2005.

I really am interested in finding an answer to this.

It seems to defy logic.
 
Top