High lactose dairy can cause large insulin spikes which can set off sugar cravings. Cow's milk casein (protein) can cause inflammation and irritate the gut, also the exorphins released can be addictive, causing highs followed by lows with sugar cravings.
There's a lot of individual variation. I find that low-lactose goat's cheese and butter aid fat loss and muscle gain.
F 5 ft 3. HW: 196 lbs. Primal SW (May 2011): 182 lbs (42% BF)... W June '12: 160 lbs (29% BF) (UK size 12, US size 8). GW: ~24% BF - have ditched the scales til I fit into a pair of UK size 10 bootcut jeans. Currently aligning towards 'The Perfect Health Diet' having swapped some fat for potatoes.