Don't feel bad, I do the same thing all the time.
At the public range that I go to, the bolt must be locked back when the rifle is not in use. Half the time I go to shotgun the action open to re-lube the carrier/bolt, I forget to lower the carrier and have the puzzled look on my face until I hit the bolt release.