Vision is specific to an individual. A rifle that is zeroed for me may not necessarily be zeroed for you.
A sight that is mechanically zeroed (i.e. centered vertically and horizontally) may not be zeroed
for you because of
your individual sight characteristics.
I always have to adjust the sights a bit whenever I'm handed a strange rifle.
So...
1.) The "baseline" is wherever the sights are after the front sight post (elevation) is adjusted and the rear sight (windage) is adjusted so that you're hitting where you want to at a specific range with a specific load. Set it and forget it. You should not be messing around with windage or elevation unless you want to re-zero with a different load (or you're making adjustments to compensate for a drastic change in range or crosswind).
2.) The rifle's shooting fine. You just need to adjust the sights for
you. Or, to put it a different way, the human is the defective part.