I put mine in in the morning shower (helps with "leaking" from residue that might be in there) and take it out before bed. I find it uncomfortable to wear overnight because it pushes on the walls for too many hours. I just wear a pad at night. I LOVE not having to take my purse to the bathroom at work anymore!
Also, why are you taking it out just because you're going to the bathroom? Thats not needed.