As this is always hard to accept, the fact is if the weight doesn't shift you have to cut back on everything until it does. I think we easily convince ourselves that because a food is good/primal/low carb that we don't have to restrict. For every person who doesn't have to watch calories, fats, portions--especially females--there are hundreds who do. I can't eat unlimited calories or fats, and from what you've written, you probably can't either. There are hundreds of posts on this site that support this. I wish you all the best of luck.
This time, like all times, is a very good one, if we but know what to do with it. Ralph Waldo Emerson
Any given day you are surrounded by 10,000 idiots. Lao Tsu, founder of Taoism