I've got krylon on several AR15's. The only thing I've seen oil or powder solvent do (keep in mind I don't scrub the outside of the rifle with either, but it does get splashed on the paint or leak out onto the paint - I'm an over luber) krylon is make it kind of shiny. Not exactly like a gloss finish, but the flat camo colors aren't really flat anymore where the oil has leaked out of the receiver. I use 3n1 oil and Mobil 1.
Try some different stuff yourself. If you feel the paint gets damaged that's a good reason to try a different pain scheme.
Now, I have removed krylon with carb. cleaner and a tooth brush by spraying the paint and quickly scrubbing while the carb cleaner was still wet. Then it takes a rag to wipe off the residue. I stripped off a carbine stock to remove the krylon when I replaced it with an A2 fixed stock on the vARmint rifle. It didn't appear to harm the plastic of the carbine stock at all and that stock in on another rifle right now (don't remember which one but it's been in use for several years after being stripped). Carb. cleaner will damage some plastics and rubbers though, so be careful if you use that stuff on a gun (or anything else that isn't 100% metal.)