The hammer pin is held in the center by a "J" spring that runs vertically through the hammer. It sits in a recess in the very center of the hammer pin.
The trigger pin also has recesses, which are held in place by the 2 legs of the hammer spring.
Double check to make sure your hammer spring legs are pushing DOWN on the trigger pin. They should be holding it in place, keeping it from moving side to side.