Most of the items are basically primal for sure. Sweet potatoes Mark says to avoid in the book, but if you are able to eat them and stay within the carb range you are shooting for (50-100, or 100-150, etc.) then I say eat up! Bananas and apples as well--I consider them totally primal. Yogurt I'd try to get full fat and unsweetened (dairy intolerant so don't do it though ), coffee Mark drinks daily fwiw, cheese I'd personally try and get raw if at all possible, but again I'd eat it (and most dairy) in moderation.
For more from Mark on all this, check The Definitive Guide to Dairy | Mark's Daily Apple on the yogurt and cheese, Are Potatoes Paleo? | Mark's Daily Apple on potatoes/sweet potatoes, Caffeine Talk | Mark's Daily Apple on coffee, and this The Best Low-Carb Fruits (and the Worst) | Mark's Daily Apple on fruit.