I have a new upper on the way and have REALLY fallen in love with the MOE SL handguards. One side effect of the SL is needing to get rid of the front sling loop, and I've never done that myself before (all my uppers with them have been factory installed SL guards). My question is, if I don't have a vice, or a wheeler block to keep the front sight post stable will I drift or cant the FSB by tapping out the rivet with a punch and hammer? I can always try and support it with a thick towel or what not, but I didn't want to risk it if I had a chance of screwing the fsb up tapping it out.