Assuming you have a standard FSB, and not a gas-block:
This happens when the slot in the upper receiver is notched a little off center, or when the FSB is not fully aligned with the nub on the barrel extention. Hence, your front sight is "canted" to the left... making you have to adjust your rear sight all the way to the left.
The fix is... to remove the barrel, and slightly remove metal from the upper receiver (using a small round file), where the barrel extension pin sticks into the receiver. Then, using a center punch, stake the left side of the receiver slightly, to "push" the pin on the barrel to the right. Then, re-assemble the barrel nut and upper.
You should be able to "eyeball" the canted FSB, with the carry handle or flat-top.... or using a laser bore sighter comes in handy for this process. in any case, the goal is to *rotate* the barrel to the right, in the upper receiver. Make sense?
It sounds a little difficult, but its really simple. Just go slow. You can get the rifle where it is dead on and the rear aperture centerd, with a little practice.