Simple - Toss it into a Monarch CNC machining center and program the machine to remove the fence.
Seriously, I think the best way with standard home tools is to emphasize hand work as much as possible. Go ahead and use a dremel or whatever to take off 80 to 90% of it, but use caution not to use power tools when you get close. Switch to files, take off another 10 to 15%. For that final bit, I'd start with 220 wet/dry paper backed by small files or steel blocks. A good set of needle files in various shapes will help a lot towards the end of the job. When you are truly done, a bead or sand blaster will blend the area and prep it for moly or anodize.
The overall idea is to keep the flat surfaces flat, and if you attack it with a handheld dremel or even 220 paper in your fingers, you are probably going to dish and/or gouge areas you don't want to, unless you limit the power tool use to the really tall portions.
You can protect (somewhat) flat areas that you don't want to touch during the power-tool phase with fiberglass tape. You can see if you miss a bit as the tape scores or gouges. It does help a bit. Of course, remove the tape when the time comes to blend.