That's like asking if roses are red. Some are, some aren't. There are many dairy farms that have poor practices, but then there are some who treat their bovine with the utmost respect. The onus is on us to do our due diligence and find which farms are ethical and give them our money. Same can be said for buying fruit and vegetables. There's monocrop harvesting, which I'd argue is super unethical on many levels as well as genetically modifying crops (no one asked the crop if it's OK to splice their genes) and then there's permaculture and hydroponic culture etc. It's one's job to source their vegan food from ethical sources too.
Since you do apparently want to continue the conversation... I think we're looking at multiple levels of ethics here, and it gets very complicated.
Is taking cows' milk ethical? Arguably not. Keep in mind, though, they do it in India, among people who consider eating beef wrong because cows are sacred. Some of those same people do not eat any meat at all, on similar ethics grounds to what Ukpoohbear is putting forth, only in that case, it's their religion that's telling them so. Others do, just not beef.
One of my good friends belongs to a sect of Hinduism that requires almost pure vegetarianism: no meat, no eggs, but dairy is allowed. So she eats vegan except for dairy. I don't know what the ethical reasoning is, or even if it is ethical reasoning or just religion. I suppose I could ask next time I talk to her.
Land use ethics are another question. Also valid.
At the end of the day, we can only make the best choices we can with what resources we have. Not everyone has the means to eat a strictly vegan diet. Some people can't because of their health. Some can't because of money: vegan substitutes in a meat eating, milk drinking world get prohibitively expensive. Some don't have enough vegan foods available to them. Some don't have the knowledge it takes to eat a sustainable vegan diet or the time to acquire it.
If none of those can'ts or don't haves apply to you, and you feel being vegan is the most ethical choice, by all means, be vegan.
One quibble I have with veganism is exactly that point about sustainable farming and land use: in an industrialized food system, the foods that make for a healthy vegan diet are not produced sustainably, at least, not all of them are. To be fair, animal products aren't often produced sustainably either. However, sustainable organic farming necessarily includes both livestock and vegetable crops. If they're not both raised on the same farm, then the vegetable farmers have to truck in animal manure for fertilizer. It's really not just one or the other.
A sustainable farming system wouldn't produce as much meat and dairy as factory farming does. Eating sustainably would mean animal products could not be central to the diet. And it probably would mean eating vegan some of the time. Just not 100% of the time for 100% of the population.
All that is my own personal opinion, not speaking as a moderator.