Calories do matter, but nuts tend to be sneaky because they leave you hungry and make you end up eating more calories (for some people!). When I was new, trying to quit soda and snacks, I used a LOT of nuts, mainly almonds, to replace basically every snack I had. So I walked around with a feedbag of almonds for a few weeks and actually gained a bit of weight - but I also became more efficient at digesting fat (which, most would say, I hope, makes the body better at utilizing its own fat), and when I got serious, it was easy to quit me feedbag habit and the weight melted off. Knowing what I know now, I'd rather have done that with bacon or something, lol. Probably would have become a better "fat digester" in shorter time.
For maintenance, I only eat nuts once a week on grocery day for the most part. That's how fattening they are to me.
Steak, eggs, potatoes - fruits, nuts, berries and forage. Coconut milk and potent herbs and spices. Tea instead of coffee now and teeny amounts of kelp daily. Let's see how this does! Not really had dairy much, and gut seems better for it.