Over the two years that I've been following a Primal lifestyle, I've noticed two things that get in the way of fat loss: nuts and dairy. I believe 4 ounces of nuts is far too many in one day. Try 1 ounce a day if you have to have them. Cut dairy if you haven't already.
As long as you eat enough protein, work out hard and get enough rest, you won't lose muscle. Try cutting out the nuts and dairy. Up the protein in the form of meat if you feel hungry. If after a month or so you don't start to see the changes you want, then reduce your fruit intake. Be patient.
And don't forget the IFing.
You lousy kids! Get off my savannah!