You're still pretty new to primal, right? I felt like that for 6 weeks before I adjusted -- and that was at 125-150g carbs per day. Eek! I powered through it on sheer will and came out the other side feeling strong again.
There's nothing wrong with taking an extra recovery day or two. Just don't let it become ten or twenty.
Have you tried a miniature carb boost, like a banana or sweet potato about 2 hours pre-workout?