No, you don't exactly need to eat vegetables. All the nutrients can be better obtained from animal sources, anyway. Eggs, liver, shellfish (oysters, esp.!), plus some fruit and nuts pretty well have you covered. BUT there are supposed digestive benefits, resistant fiber, etc. that are often toted around here. But if they are causing you pain, then they are clearly not worth it at this point. Especially since you won't be digesting them well, anyway!
Steak, eggs, potatoes - fruits, nuts, berries and forage. Coconut milk and potent herbs and spices. Tea instead of coffee now and teeny amounts of kelp daily. Let's see how this does! Not really had dairy much, and gut seems better for it.