Many people who meditate go through this. When the mind or body tries to still itself, there is sometimes a panic that is then an anger. It's okay! It's just something to work through.
You know what? I practice kundalini yoga, and lots of times we chant, which is great for grounding and gives you a focused energy point that goes into the collective, vs. going inward. Snatam Kaur has a beautiful CD called GRACE. You can google it. Try Ra Ma Da Sa...if you can sing it, you can breathe it, and it won't feel like "work" to meditate.
Best Wishes,
Sandy