Is your milk fortified with vitamin A by any chance? Too little sunlight (Vitamin d) and too much vitamin A can cause problems for your skin.
I ask because you said being in the sun gets rid of it. Even small amounts of vitamin A can antagonize the effects of vitamin D.
