This is why I like the archevore.com explanation. It's not about a paleolithic reenactment fantasy, it's more an avoidance of the neolithic agents of disease. His 12-step program puts the whole dairy issue dead last, number 12:
12. If you are allergic to milk protein or concerned about theoretical risks of casein, you can stick to butter and avoid milk, cream and soft cheeses. Aged cheeses 6 months and older may not have beta-casomorphin and are good sources of K2.
Female, 5'3", 49, Starting weight: 163lbs. Current weight: 135 (more or less).
I can squat 180lbs, press 72.5lbs and deadlift 185lbs