Here's the scoop. Colt made a few changes from mil-spec in the past on some of their rifles:
1. Large pivot pins. If you have a large-pivot Colt lower, you'll need a conversion pin to use a small-pivot upper, and the opposite holds true as well.
2. Large firecontrols. Colt increased the size of their hammer and trigger pins from .154" to .170". Obviously, these pins and the firecontrols associated with them will not interchange with standard mil-spec parts.
3. Sear blocks. Colt put a block in the rear of some lower receivers years ago that sticks up out of the lower. This is so that you could not use an M16 bolt carrier in your rifle. If you have one of these sear blocks, you will not be able to use standard AR-15 bolt carriers.
Everything else interchanges just fine.