I am a current avid Crossfitter and beginning weight lifter (yes, I separate the two, because I am tired of fighting the good fight that some of us Xfit kids are dedicated to weights, ;-) ). I have had some notoriously difficult workouts in both disciplines, and have also done some cardio challenges. I even did a couple of 12 mile mud runs that really tested the hell out of me! Hands down though, the best I have ever felt after any workout was when I was doing intramural swimming in college.
I didn't know any of the strokes, so I joined a beginning swim class, got much better them, and worked my way into the intermediate/advanced class. By the end of a year and a half I was doing 3000 yards a class and every time I left, I just felt exhausted and awesome. If you have the time and dedication, swimming, I think, is the way to go. If you aren't a strong swimmer, do a beginning class and progress. Once you know all four strokes, at least the basics, you can even join a master's program at your local pool where someone will do the programming for you. Hands down the best exercise ever, and low impact great for your joints as well.