My hypothesis that there is a small percentage of population that have fantastic (or different) blood sugar control and extremely low natural appetite. Those folks are blessed by ability to adhere to a raw diet/vegetarianism. They would be slender no matter what they'd eat, but they have a luxury to chose the tastiest foods on the planet and eat just a bit of it.
I believe people without these genetic modifiers simply get too unwell to adhere. I know for myself sugar gets unbalanced quite fast without stabilizing impacts of fats and proteins, resulting in insatiable hunger and craving for cooked food with drier texture, like bread. I know that because when i was young we'll go to the seashore on vacations and eat fruit by the bucket (not available most of the year in any variety and quantity in the former USSR), and after a few days, all you start wanting is bland bread, like flatbread.