Welcome, ebrady. Sounds like you are doing well.
Forty minutes of walking is great exercise.
Sugar is not great, of course, but when you have some, enjoy it and move on.
DO NOT CUT FAT OR CALORIES! Just cut carbs until you have lost the amount of fat you want to lose.
Good luck with Primal!