Colt does not recommend replacing the front sight tower. The a new tower is extremely difficult to align properly. Holes won't line up, barrel won't be square to upper, etc. Factory tower is aligned, reamed and pinned at the factory. Armorers school teaches us to replace barrel assembly if sight is bent or otherwise trashed.
Pins are a pain in the ass to knock out. You need a heavy taper pin punch and a 16 oz ball peen hammer plus lots of padding for barrel. Reinstall with new pins.