Having a tendency in the other direction and having the ability to eat like a 25 year old male construction worker, I can't trust my intuition.
When people would ask if I were hungry, I used to joke that I'd been hungry since I was 12. Except, that it's not all that funny, and it's close to true.
Hard exercise has never made me feel more hungry (except sometimes after swimming), and has often made me feel less hungry. Though a lot of people seem to feed themselves more on workout days, I don't necessarily agree that the added activity of the day "naturally" makes you more hungry. If you think back to before we had what I think of as artificial exercise, if a person were out hunting or migrating, it would make more sense (to me anyway) that the body would dial down hunger until the person settled in to a place of rest.
While there are times when listening to one's body makes sense (you don't want to lift a heavy thing until you hear a crack or a pop for eg.), I don't think that the artificiality of our environment is conducive to most of us getting the hunger signal only when we're truly in need of sustenance. Go into a movie theatre and you want popcorn because it smells like popcorn. Walk down a busy city street, pass street after street of raw oyster signs, and you may just finally pick one and go eat a dozen raw oysters. TV commercial after TV commercial of a given food may trigger you to want that.
So, from the opposite perspective, I can tell you that "listening to my body" when it comes to the consumption of food has only ever made me fat. Which doesn't mean that I'm recommending that you force food. But I do recommend examining your relationship with food. If you think of it as evil, you might be in trouble. If you think of it as fuel for the things you want to accomplish and also as the thing you need to have beautiful skin, hair, and good health, you might start wanting a bit more of it.
"Right is right, even if no one is doing it; wrong is wrong, even if everyone is doing it." - St. Augustine