Yes. Your rail is fucked and out of spec. Seems to be machining issues. I.E. a dull bit pushing material out of the way rather than removing material, creating a burr on the edge of the cutting terminal.
How this works in regards to your scope mount, is the mount applies a certain amount of pressure to the upper that when full surface area is in contact, appropriate purchase is achieved and the optic does not move during firing.
Apply that same pressure to a significantly reduced surface area incapable of maintaining form or friction under that amount of pressure the mount applies vs. inline forces of the cycle of the weapon, and it will not generate an appropriate amount of surface tension/friction, allowing the optic to move when enough force is applied (object at rest= scope, object in motion=rifle: scope will attempt to remain at rest until hard stops engage and scope is forced to move). This happens on both the rearward stroke(unlock, extract, eject, cock) phases of the cycle of function and during the forward stroke (load, chamber, lock), so it uses the full range of motion the specifications and tolerances allow.
As this happens, the smaller surface area experiences wear and deformation, thereby negating applied torque value during installation.
Bad news, the upper is out of spec.
Good news is that as the high points wear down and full surface area contact is achieved, this will stop happening. If you don't mind cosmetic issues, you'll eventually have a working upper capable of holding zero.