We have encountered this, especially with out of spec parts.  1st buy quality.

As to the fix.  With the selector in place what ya got is essentially a blind hole.  Remove the upper and then give the lower a few good taps with a dead blow plastic hammer and see if you can get the detent to drop down a bit to allow you to remove the selector.  Do this in a vise and wiggle the selector whilst hitting the lower.  Usually they will come out with some patience.  Oil up the hole real good with some Break Free or G96.  

This is an ugly problem.

Looks like I untacked that thread too soon.

Tap the lower on a firm padded surface until the detent clears the shaft of the selector. The remove the selector and push the detent out from the top.