So I done 5 lowers already all come out flawless the only problem I have is everytime I drill the safety selector holes like they say to do one side then flip and do the other side after you mill,my safety selector won't fit all the way unless I put it back in the jig and drill all the way through one side to the other and then it works perfect.Does anyone else have this issue?

Means your jig/table isn't perfectly perpendicular to the drill axis. The final through hole is making the two holes slightly oblate, but at least on a common axis, so the selector can pass through cleanly.

When I drill the safety hole, I do it before I mill, I drill 1/2 in on each side and then mill the pocket, have not had any problems at all with getting the safety selector in them.

Same issue here

So today I drilled and milled another one but b4 I did the safety selector hole I moved my drill press to our patio table lol it's more sturdy and was more leveled but in the end it was the same Damm thing???? I had to drill through one hole to the other so it'll fit so it must be my drill press huh?

Something is off, might be the drill press, but if it was runout, I would expect the hole to be to big and showing on the jig, I  have a set of needle files I use if I can't get one in, I just use the needle file and touch it up until it fits, normally when I have to do it, takes less than 5 minutes.  That said, I have only had to do it a couple of times out of a dozen that I have milled and built.

I use a Modulus jig, and I find that I have to use a 4mm cylindrical polishing stone (on a Dremel) to get the selector switch to feel just right.

I start by running some 400 grit sand paper (glued to a dowel) through the holes (one side at a time) just to smooth them up.
I then start fitting the selector to see where it binds up (and it almost always does).

I polish the bearing surfaces of the selector switch until they are perfectly smooth and shiny.
By looking to see where it binds up as I insert it into receiver, I can then polish up the holes with my Dremel.
I do a little bit on each side until it fits just right.

Takes about 30 minutes.

G.