fierces reasons you need dairy are as much woo as the reasons you in your post saying you need to avoid it. Dairy is an individual beast that you can live with or without. Just figure which works better for you.
Except if you're an infant or want to get swole. In which case you need to drink breast milk and GOMAD, respectively. Just kidding about the GOMAD.