Yeah, not that most people in the Old Stone Age probably did - live in a cave I mean.
They almost certainly used temporary shelters, like modern hunter-gatherers. You can't be static if you have to follow game. However, it's true some at times utilized the mouths of caves, or rock overhangs, like Mr. Suelo.
What caves seem to have represented for them - or some of them across that vast period, at least - is places of awe and deep significance. In France and Spain there are cave paintings in almost inaccessible places, where you could only see them by flickering torchlight, if you could get there. Were they even painted to be looked at? is an interesting question. There are probable connections with the "other world". Also there are hints of connections with the female body (hence fertility and birth?): at one site there are three adjoining caves with strategically placed rock-carvings that strongly suggest this. It seems beyond doubt (because of certain features of some paintings) that there's a connection between the paintings and altered states of consciousness - seen as highly significant in many societies - which isn't surprising since we know this was true for the Bushmen, who were still making such paintings not so long ago. In short, more like cathedrals than houses.
It's a remarkable human-interest story in its own terms. If he really has "decided to stop using money", as is stated, I wonder how he gets that clothing he's wearing or that beans and rice in his cooking pot. Does he beg for it? Does he do odd jobs for people but ask to be paid "in kind" rather than in money? What does he do, if he breaks those spectacles he's wearing? I don't know an optician that accepts payment except in money.
Not using credit cards or debit cards - "plastic" - would be unusual enough these days - and not doing so is the only way to drop "off the radar" in a modern society. But refusing to use money seems harder.