The first thing you have to know is the level of contaminants you're dealing with. Have you had your raw water tested first?
You might have naturally occurring contaminants. You could have iron, calcium, tannins, sulphur, organics - just to name a few things. Then there's the bio hazzards that may or may not be present like micro organisms (bacteria), cysts, etc. Then finally, there are the man made chemicals that could have leached into your well.
I don't know if they still do it, but Sears used to accept water samples that they would have tested. If not, there's bound to be a lab around somewhere that does. If you can't find a place on your own, check with some of the local water treatment companies to see where you can take your sample.
Whole house carbon filters work great in the right application. I have one on my water system, and it has made a tremendous difference.