If the lower is a large hole, which I'm assuming it is, there should not be a problem. I had a similar circumstance mating a Bushmaster small holed upper to a Colt large holed lower. The combination fit well.
However, I had one combination where I had a large holed SP-1 upper mated to a Rock River Arms small holed lower. The bushing that mates the two, makes the combo very tight.
Either way, you should be good to go.