OK. Now I see it. You've managed to insert the magazine past the catch - which is hard to do... First thing to do is get rid of the live rounds. That's a dangerous condition. Either pull the floor plate off the mag, or remove the rounds carefully.
Now remove the mag catch. I assume you know how to do that.
Magazine should come out - but you may have to exert some force on it. Rock it forward first and see if it'll come out.
You need to figure out how this happened. I've looked at my rifles and then pulled out a stripped receiver. It isn't possible to insert a magazine like that unless you REALLY try. It *should* stop on the bolt catch at the very last. Either the magazine is somehow bad, the bolt catch is somehow bad...
I've even dropped rifles on their magazines and I've never had a stoppage like that. There's a defect somewhere.
Another thought... Were you pressing the mag release button when you inserted the new mag?
No matter what the cause, get those live rounds out of your way, and then remove the mag catch. That should do it. Let us know what you find when you get it apart.
Shane