The biggest reason to not go 0% fat in some foods, especially dairy, is that when the fat is taken out, it is often replaced with carbs and gums. Fage 0% doesn't have garbage added to it, and I'm pretty sure it's not the only one. Read the labels always to make your choices. If you see the carb grams going up as the fat grams go down, the company is probably adding something to make the fat free product more palatable. Nonfat milk is not inherently bad - a nonfat lifestyle is.
I like 0% fat yogurt just fine. But not for cooking as it will break too easily.
I don't know if you can meet all your nutritional goals on yogurt and plants. You might want to plug what you're eating into a one of the online food trackers to see. If PA is one of the states where raw milk is legal (even if only for animals), you also might want to make your own yogurt.
Though Mark has laid out the specifics for Primal, and they are definitely skewed toward eating a lot of animal, they are also about eating no poisons (no poisons on your list) and not eating the CW recommendation of carbs. Carrots, bananas, some melons, and mangoes have a lot of carbs.
Bottom line is if you feel good and you're not substituting fruit to get the sugar you miss from candy/cookies/etc., you're probably okay. JMO.
"Right is right, even if no one is doing it; wrong is wrong, even if everyone is doing it." - St. Augustine