I have the same reactions to it - in fact, I'm pretty much asthma-free without it, not to mention without sinus congestion, a stuffy nose and itchy ears.
The only thing I've been able to tolerate so far is ghee in very small amounts. I've gone the goat's milk route (kefir) as well as sheep's milk (yogurt) and still had issues, though they were much milder than my reactions to cow's milk. Finally accepted the fact that I'm much better off without dairy, period. Besides, coconut milk does a body good.
Life is not a matter of having good cards, but of playing a poor hand well.
- Robert Louis Stevenson