I can't get that thing to budge at all... what's the trick?

It's a noveske SS barrel/vltor gas block

I've never personally had a problem getting one out. It might help to soak it for a couple days with penetrating oil- Kroil is good. Make sure you have the right size punch and it should come right out.

Which side are trying to knock them from. It could be a tapered pin.

I just got mine off from the Noveske barrel I just bought.  I put some 3 in 1 oil on it and let it soak for a couple of hours.  Leave the set screw in when you go to drive the pin out and put the screw in first before you drive the pin back in.  This helps keep the block from rotating around the barrel when the pin is almost out.  It's not tapered and can be driven out from either side.  Put some electrical tape around the pin, the finish on the block scratches pretty easily.  Use a bench block or some scrap lumber to support the barrel an block.  I had my daughter help hold the barrel steady so I could use both hands to steady the punch and wield the hammer.  I used a flat pin punch I got from Lowe's that was the right size for the hole and gave the punch some solid hits with an 8 ounce hammer.  Took a couple of minutes to get the pin out and then removed the set screw.

I got it out.  I let some WD40 sit on it for about 20 minutes.  It took quite a bit of force to get it out, but it popped right out np's.