Hey, if the brake is a Shimano canti, like the one in QC's picture, Shimano has had a long standing recall on those. The grey plastic casing with the spring in it cracks and screws up the spring tension, making it so the brake won't center properly. If this is the case, which it sounds like it is, go to you local shop with the grey plastic springs and ask for new ones, if they don't have em or want to charge you for them, IM me and I'll mail you out some new ones.
To get the grey plastic spring (collar) off there should be a 5mm allen or a 10mm hex head holding the brake arm on. Loosen that all the way out, and pull the brake arm off the brake boss. Next pull on the grey plastic piece and it should come right off. If for some odd reason that does't cure it, take some pics and I'll see what I can do. Hope that helps.