Technically no, having small amounts of food is no big deal in grand scheme of things, but I find it actually makes things much easier in the long run. IFing while having small bites of food here and there is like trying to eat Paleo with small amounts of grains and sugar on a daily basis. You just end up resetting your progress every single time.
And - at least this has been my experience - you'll find that going long periods of time with absolutely no food feels really good, and you won't want to stop. Just keep your mouth occupied with plenty of water, green tea, iced herbal tea, yerba mate, and/or black coffee, and ride that catecholamine wave.
“The whole concept of a macronutrient, like that of a calorie, is determining our language game in such a way that the conversation is not making sense." - Dr. Kurt Harris