Not eating after a workout is not going to do you harm, but never eating after a workout might not be a good idea. Like wise, immediately refueling after a workout is not going to do you harm, but always doing it is not ideal either.
i would say mixing it up is the name of the game. Pick certain days to eat breakfast afterward, and go without on other days. Mix up your eating patterns as much as possible. Our bodies are dynamic and adaptive and will quickly get used to a routine, interfering with our wishes to lose fat and become stronger.