No need for dairy, it's just convenient. So that's easy to avoid.
As for cholesterol, mostly dietary cholesterol doesn't influence your cholesterol. The doctor doesn't know what he's talking about.
As for energy and carbs, there's no problem with carbs as long as they're from primal sources. So mostly tubers and some fruits. White rice is no more or less simple than potato, so I see no reason to prefer rice over potato.
"Thanks to the combination of meat, calcium-rich leaf foods, and a vigorous life, the early hunter-gatherers were robust, with strong skeletons, jaws, and teeth." - Harold McGee, On Food And Cooking: The Science and Lore of the Kitchen