I have looked into this issue a little, and it seems that there is no settled science. There is some evidence that BPA, a common chemical in plastic containers, can cause issues, and that plastic molecules that get into food from heating plastic containers can cause problems at the cell level.
There is enough uncertainty that I decided to go along with what you said: better to be safe than sorry. So, I threw away my plastic food containers and bought glass containers from Pyrex. They do have plastic lids, but without BPA; I remove the lids and cover the containers with paper towels when microwaving food in them. I also hand wash the lids, so that the high heat of the dishwater doesn't break down the plastic over time.
Live your life and love your life. It's the only one you get.