I found PB because of a seizure disorder so I have done a lot of research into the neurology of this diet. If you imagine the neural pathways as electrical wiring, fat is the body's black plastic coating on the outside of the wires. In very fat poor diets, when there is wear and tear damage to neural pathways, you don't have enough fat to patch up that sheathing on the outside so the electrical impulses can go astray, in your case forgetfulness, in my case, epilepsy.
As they say, if you don't use it you lose it. Sounds like you are using it well. Have you tried sudoku puzzles? I find them a delightful way to keep the old gray matter ticking over.