I've experienced this with Olympic lowers before.
The recess where rear lug of the upper receiver fits into is sometimes to shallow on Olympic Arms lowers. This, in turn, causes a very tight fit for the take down pin. In the past I have "fitted" the lug on the upper to my lower. However that lug butt-ugly. An easier way to loosen up those pins is simply to use a round dremil stone to "open up" the hole in the rear receiver lug.
Warning, don't do that before you determine if it's the fit of the upper and lower or if it's you pins. Sometimes those take down and pivot pins freeze, because they're not machined correctly and the detents engage the pin too deeply. To check this all you have to do is to determine if the pin is too tight during its whole movement or just at the beginning of its movement.
With the rifle disassembled, if force is needed only to start the pins, then it's the pins.
With the rifle assembled, if force is needed to move the pin during the whole course of its travel, then it's the upper / lower fit.
I've never had problems with upper and lower fit in regards to RRA, Colt, Armalite, Bushmaster or Eagle. Granted RRA lowers may be a bit tight at first, but they wear in nicely. I've had ASA lowers with strange "canted" and "mis-indexed" pivot and take down pin holes. And I've had a whole boatload of Olympic lowers that had to be fitted to uppers.