The sling swivel should be held into its base with a roll pin (if it has a rivet there, it's not original).
If it does have the correct roll pin, just tap it out with the appropriate sized punch.
If someone replaced the roll pin with a rivet, like the ones used on the front swivel of newer ARs, then you will need to cut the flared end off with a pair of diagonal pliers (dikes).