The threaded end of the mag catch assembly should be flush with the face of the mag release button when installed correctly.  If you find that the end will not flush with, or ends up up a 1/2 turn short of proud of the face of the button, then pop the button out and rotate it so what used to be the front edge is now the back side, and reinstall.


Also, before you put the button it, check the channel seating edge and the mating side of the release assembly edge for burs that may not be allowing the catch to seat all the way in to it normal at home position.

Also, before you even install the mag release button, look up the front wall of the mag well, and make sure that the upper receiver front edge is not hanging over the front wall of the lower receiver mag to prevent you from being able to push the mag all the way up in the well.  The take your mag with the bolt locked home, seat a mag as deep as you can, then look into the side slide of the bolt catch to make sure that the slot that the catch is going to lock into on the mag is actually getting high enough for the catch to lock into.
Hence some mag lips are thicker and will bottom out against the bottom of the carrier before the slot catch point is high enough for the catch lip to actually lock into as well.

As for once the mag catch is installed, just look into the mag well to determine if the hook point of the catch is protruding into the receiver deep enough.  Here short of a bur, could be the lower receiver home shelf was not milled deep enough for a catch to seat home correctly, or just the catch seating surface not milled deep enough instead.