It sounds like you're really on the right track. You're lowering fruit and dairy and upping the fat. I guess the base is vegetables, meat (including fish and fowl) and healthy oils. Anything else is sort of an extra if you're trying to lose weight. Nuts cause plateaus for some people, as does fruit. Blueberries are one of the best choices if you're going to have fruit -- but I do believe the basic three will give the best results with an (IF -- skipping a meal if you're not hungry now and then). It is great fun to tweak it to make your own personal PB that works for you. It must be fun and not stressful. That's huge.
As far as co-workers and other people you don't live with go, just say you have celiac or sugar issues, or, the easiest, just don't discuss your new lifestyle. Often times people don't want to hear about good things from other people when they're messed up. You don't have to eat with your co-workers, do you? Anyway, everyone understands allergies to wheat and sugar, so, that's a good excuse. If it's causing your body to be fat, it could be considered an allergy:-):-):-)
Your flatmate won't disapprove too long when he sees the wonderful results -- better sleep, a fitter body, etc. If he's complaining because he's not losing and you're looking better and better, then, he has a couple options -- start primal or -- maybe, well, I'm going to let you draw your own conclusions there:-)
Think about the feel of looser shirts
Keep it simple
Eat really tasty food