Couple of things come to mind. You may not be fat-adapted, or perhaps your body fat is already too low. I have been IF'ing for months, and now that my BF is getting very low I'm finding it tougher to keep cruising along for my usual 20 to 22 hour window.
Alternatively, try eating before bed instead.
Steak, eggs, potatoes - fruits, nuts, berries and forage. Coconut milk and potent herbs and spices. Tea instead of coffee now and teeny amounts of kelp daily. Let's see how this does! Not really had dairy much, and gut seems better for it.