Again... the determination is made by the person at the receiving end, not the person at the dishing out end.
Well, if someone is offended by a joke, then obviously you shouldn't joke about that subject or in that way around them again.
But I do think there is room for people to get offended unintentionally because of their mood or how seriously they take a given subject. One person might take offense to something that wouldn't offend anyone else in the world.
I think that it becomes trolling once you become aware that it gets to them, and then you keep doing it. That kind of behavior is pretty much inexcusable.
However, I have seen people "playing the victim" before, trying to get people in trouble by feigning offense at something that normally wouldn't bother them, simply because they dislike someone. I consider that equally unacceptable.
I don't believe that the person on the receiving end of a joke should have the absolute and final authority to brand someone a bully or a troll, and then assign consequences. There should be someone impartial with the authority to decide that on a case-by-case basis.
I mean, certainly an argument could be made that no one should ever joke, that people should be completely serious and literal about everything because there's a risk of offense. But frankly, I'd rather not have to word all of my interactions with people like legal contracts.