It is and it isn't.
DS has any dairy he wants. He usually gets raw yogurt and organic cheddar cheese. DH and I consume butter, but everything else is "special occasion."
At our winter celebration (in June), we get some fancy cheese and eat it. We love good cheese. And then we have gelato every now and again (more in summer -- so now -- than the rest of the year).
Otherwise, we only use butter regularly (daily, really), but all other dairy is off the menu. DH is working through some gut issues, and I'm a bit lactose intolerant. Besides that, it's ridiculously expensive here in NZ. And by that I mean insanely expensive for cheese.