In my opinion no. You are using PWO carbs to replenish your glycogen stores because of the constant training. Limiting your fat intake WILL lessen your total caloric intake but fat helps lessen the insulin spike you will get from the extra carbs. Feed the Machine! Any of you PB'ers please correct me if I'm wrong.
Also I would recommend Sweet Potatoes and/or fruits as your carb of choice. Def. the lesser of all the carb evils.