this is a tricky one for me too. i would consider myself very fit compared to most of the people i know, but, like you, likely way below the nomadic hunter/gatherer. i know that i'm not as fit as i would like to be.
but, i think fitness is measured by your ability to move yourself effectively and continually through space. i can hike all day without external fuel, run a few miles, lift myself/climb trees/rocks/etc. with relative ease, sprint/jump effectively. so, does that make me fit? yeah, but i could be a lot better at all of those. i guess i should try some persistence hunting and carry back a "carcass" to see how i really measure up.