FWD-
Worst thing? Torque steer just plain old sucks
Hard to accelerate out of turns when the car is both pushing (understeer) and fighting you. Eventually you'll have to replace CV joints but not for a long time. Steering effort feels fairly disconnected from road.
On the good side they generally have good grip in snow (my Honda and VW did anyways), good directional stability when you hit a wet patch in the road and hydroplane so it's pretty unlikely to get out of control like an RWD car. Generally most inexpensive/economical option.
RWD-
Much more fun and lively if the car has any power so you can really play around. Steering usually feels more connected and precise since the front wheels aren't trying to drag the car around and steer. Better performance under acceleration.
Cons are that they'll generally be terrible in snow or wet conditions, with the more power the worse off you'll be. Hydroplaning can quickly become life-or-death as the rear of the car will have a tendency to get away from you.
AWD-
take a ride in a Subaru WRT or Audi S4 and it won't feel sluggish. I think this is more a function of the specific car than anything else but AWD does add quite a few pounds (maybe 400) to the car so it will weigh more. The best thing is stomping it from a dead stop on a wet pavement, the thing will just scoot as all wheels have grip. Also there are a lot of AWD cars out there with sophisticated stability control systems that will do everything physically possible to keep the car on the road as long as you aim it in the right direction. Very cool in my opinion.
Cons are that it will generally be more expensive and there really aren't that many cars out there to choose from (certainly not like FWD or RWD). Long term maint costs may be higher?
Given a choice with everything being equal I'd prefer an AWD car but since they're not equal I went with a RWD car.
I just got tired of the feel and handling of my last 2 FWD cars and went back to RWD.