Walk. A lot. It helps you lose weight and builds strength for you to start doing more intense exercises.
There are a multitude of things that could be causing it, from a previous injury to stiffness to simple weakness in the stabilizing tissue. Walking won't hurt it like running or some other exercises can.