Since any bending or flexing of a punch will dissipate the force and cause things not to move, I made my sight punches from brass bar stock.
I usually used 1"x1/2"x6" brass bar stock, which I then cut the end to fit the sight. This won't bend or flex, won't deform as badly as a small rod type brass drift, and puts all the force on the sight,
It's also easier to get accurate strikes on with the hammer than a small round rod type.
Once you move the sight, there'll be brass marks on it. Just use a good copper removing bore solvent to remove the brass marks.
Remove the slide from the frame, and if it's a tight sight, tape it up with plenty of tape to protect it.
Put it on a SOLID surface that won't allow it to bounce around. A concrete floor works well.
Put a pad of paper under it to further protect it.
For a hammer, buy a ball peen hammer at any hardware store. You don't want a really big one, nor too small.
One trick that often works very well is to buy a cheap automatic adjustable center punch at a hardware store.
Braze a glob of brass braze on the tip and file it to shape. Keep the heat from spreading to the body and spring.
Put the punch on the sight with the slide laying on a firm surface and push the punch until it "fires".
The impact will usually move all but the really tight sights.
Years ago some company sold these as sight pushers, but you can make your own.