My experience is from diving too. If you go check them out now, there are a new breed of wet suits out from o'neil(I think, they could be henderson). They are strechy(sp) 3 mm wet suits. Unlike the old binding 3 mm of just last year, these are much easier to get in and out of and have very little binding. I do not know how they would fare for swimming, as I would think just bulk would be bad, but these feel as thin as a 2 mm even though they say they are 3.
No matter what they say, gold core is still hard to get off when wet. They just go on easier.
If you need it tight for swimming, think about spending a few bucks on getting zippers installed in the tight spots so you can shuck out of it faster. It is about 50 bucks to get zippers put into the arms, so for another 100 you could have zippers put into the legs and sides to help get out of it quicker.