I'm trying to lose fat at the moment, not gain muscle, but I never eat breakfast either. I heard something about a chemical release while you sleep that makes you not hungry in the morning (I suppose this would be useful if you had to hunt/gather your food first). From what I understand, a 19 hour "fast" every day isn't abnormal. This is roughly what Dr. Harris of the PaNu blog does.
The only issue you'd have with gaining muscle, I think, is if you're not eating enough calories. It can be hard for some people to meet their caloric needs for this (sometimes 5,000k+) in one or two meals.