Chargers/Challengers are not hardcore track cars. That's not their purpose in life and it's not what most people who buy them want out of them.
They are reasonably good handling cars with some grunt that lets you have some reasonable fun when you want to and still be comfortable when you're driving from A to B. People often bitch about the weight and they do feel heavy...but if you take them out to some of the mountain roads and open the taps a little bit, you'd be amazed at how well they handle despite the heavy feel at normal traffic speeds. I've taken some of those twisty mountain-road corners we have here in Virginia at heart-attack speeds with no problems from my Charger. If you don't handle the car stupid you won't get much understeer, and if you disable the traction control what understeer you get can be fixed with the correct application of the throttle on the higher HP models. (RT and above, generally) The non-SRT models need a brake upgrade, IMO...the SRT's come with the Brembos which work well.
If you want to sharpen the handling further, you can upgrade the suspension with a number of different kits.
The Chrysler traction control computer is annoying as hell in its operation, but you can limit the stupid by turning it "off" (which only reduces it) or by using the top-secret handshake method that really disables it. The latter is fun when you're in an open paved area that hasn't been plowed after a snow and you feel like doing some drifting that doesn't shred the tires.
They're not proper sports cars...they're sporty sedans. They won't eat an M3's lunch, but they do handle fairly well and they have enough oomph for most tasks. If you want something that will be suitable as a daily driver and will still smoke everybody on a real track, the Challenger ain't it. Buy a Porche 911.
If you want a
good-looking, roomy, comfortable car that will be fun at a red light and comfortable on the highway with
absolutely atrocious gas mileage in town, they aren't a bad option.