How many times can you "rack one in" before set-back occurs and becomes a safety hazard?

I've heard as a rule of thumb anywhere between 4-6 times, but don't know if that's correct. Sounds about right.

I have a 1911 that will get set-back from one chambering though...

All you have to do is set it on a flat surface next to another one of the same lot, Right? I have seen some torn up brass from multiple chamberings/extractions, I just rotate them to the bottom of the magazine and/or shoot them on my next outing. Racking one in, from the magazine is the proper way to feed the Colt type, as opposed to dropping one in, then dropping the slide on it. Military ammo generally uses a sealer/waterproofing on the projectile, helping prevent set back, if I'm not mistaken. And a compressed load also helps, as the powder charge is packed under the bullet.