If you're 6'2" and 180 lb, then you don't need to lose any pounds. I think you should concentrate more on eliminating those "breaks in the diet on weekends." Just eat the same number of calories that you did before Primal, except replace the carbs with fat and protein. That alone may get rid of the belly fat.
5'0" female, 45 years old. Started Primal October 31, 2011, at a skinny fat 111.5 lbs. Low weight: 99.5 lb on a fast. Gained back to 115(!) on SAD chocolate, potato chips, and stress. Currently 111.