It's probably best to avoid dairy if you feel it has nothing to offer you that you can't get from other foods and if you're not prepared to conduct strict experiments with different types of dairy.
The consensus is that A2 and organic grass-fed dairy are the most healthy.
Be aware that dairy can amplify autoimmune problems for which it's not a root cause. If that's the case for you, eliminating the root causes can allow you to enjoy many forms of dairy without negative effects, and with all the positives.
F 5 ft 3. HW: 196 lbs. Primal SW (May 2011): 182 lbs (42% BF)... W June '12: 160 lbs (29% BF) (UK size 12, US size 8). GW: ~24% BF - have ditched the scales til I fit into a pair of UK size 10 bootcut jeans. Currently aligning towards 'The Perfect Health Diet' having swapped some fat for potatoes.