I bought an M4 upper from Pete with a Rock River Arms M4 side-sling adapter already installed. I've removed the pins that hold the piece opposite the side of the sling swivel on, but the curved bands are stuck under the front site/gas block. I've tried everything short of brute force including liberal amounts of CLP and trying to tap it out with a delrin punch and mallet. Does the front site need to be removed in order to remove the sling adapter?

Tap the pin out...I removed mine (RRA), it only took a few moments. There's no need to remove the front sight base.