Has the rifle seen a thorough detail cleaning since you started shooting it?
The products of the break-in and firing end up in the upper, charging handle, bolt, carrier, cam pin, barrel extension, and trigger mechanism. Some of it is gritty and can contribute to rough feeling. You want it gone.
Clean all of the rifle. CLP the bolt parts, and either CLP or moly grease the charging handle, bolt carrier, cam pin slot, and trigger, and see how it works. Don't forget to put a touch of lube on the hammer face where it rides on the carrier. Then either grease or at least put some dots of oil on the action spring.
Good luck!