Don't use the reaction rod to install a muzzle device. It's meant for barrel installation only.
You can make a cheap barrel clamp to use for installing a muzzle device, or you can do the down and dirty method by clamping the upper between your legs
Barrel clamp:
Get a block of wood about 2"x2" 3x3" or somewhere in between and about 2-3" long.
1. Make a pilot hole in the center
2. Drill the pilot hole with a forstner bit, or a large drill bit that is about the same diameter of the barrel diameter near the muzzle threads or wherever you will be clamping.
3. Cut the block in halves.
Now you can clamp your barrel in the vise with the block of wood.
If you need more grip, Brownells sells rosin that will grip the barrel to the wood a bit better for a more secure fit in the block and won't mar the finish.