Lord P, two things occur to me:
1. Maybe you should talk to a counselor and learn techniques that can help you deal with and/or eliminate these feelings. (Cognitive therapy is used for a great many things and it might help you.)
2. I've read that a part of physical attraction is because of pheromones. Google it and see if there are any answers for you there.