Physically, I don't know; but mentally if you feel you need a whole day without restrictions you're possibly setting yourself up for failure. It sounds like you're denying yourself and forcing through 6 days a week, and the 1 day off is when you're relaxed. This'll probably get tiring after a while. It might be better to train yourself not to feel as though you're missing out by eating primal.
Also how severe a departure would this 1 day off be?