I think basically every reply on this thread missed the most important point. Nobody should try to force you to eat a certain way, but that isn't exactly what I'd call this. It's just a culture mismatch.
Eastern and western cultures have different ideas and paradigms that are sometimes so deeply ingrained that we assume they are universal. Specifically, western culture is very focused on the self. When you go to a restaurant, their job is to give you what you want because you are there to fulfill yourself. Some eastern cultures (although I'm not sure about Vietnamese) view it differently and there is, for lack of a better term, a "right way" to do things. This often happens with food and drink.
I remember hearing a radio piece about a woman's visit to Japan where she wanted her green tea sweetened with honey...
"Green tea is not drunk that way"
"But that's how I like mine"
"Ah, you have been doing it wrong. We will teach you how to prepare it properly"
So, when you said you'd go get Pho with him, it's likely he had something very specific in mind. Just like when you say you'll drink green tea with your Japanese friend, they don't expect you to put honey in it. You weren't wrong, except to assume that their culture is like ours. It's a common mistake that I've made a lot when spending time with my asian friends/girlfriend.
So swallow your pride and apologize, even though you've done nothing wrong. Admit that you didn't understand his culture, explain yours, and offer in the future to try things his way. Or at the least be mindful of what he means when he invites you somewhere.