I totally agree, none is probably best. But occasionally probably isn't too bad. I think lots of the big problems come from when you've stripped the mucus lining of your GI tract and then you get rips, compromised intestinal walls... etc. But if you have a good mucus lining, you'd be more protected. And if you don't eat grains/wheat too often, then you'd be able to retain your mucus lining better (theoretically.)
The anti-nutrient thing probably isn't a huge deal if you're only eating a small amount of anti-nutrients, but they can bind up other nutrients in your food as well as just the food that they come packaged in, so they may be absorbing some of your "good food" and locking up what is available to your body.
Overall, I just don't think grains should be the *base* of your nutritional pyramid.
It should be more like the very tip top.