Jteezy,
You're thinking about it too hard while you're doing it, or maybe you are not strongly either right or left eye dominant.
Try using both hands and making a circle with the thumb and forefinger of each hand. (holding them together so you only have one circle) Make the circle as small as you can and still see a far away object at arms length. Move it back to your eyes fairly quickly, keeping the object at a distance in focus, and not your hands. It should go back to your dominant eye.
Some people aren't strongly l/r eye dominant, though. Cross dominance to deal with as a shooter. (ie; Right handed, left eye dominant)
I'm right handed, and thankfully enough strongly right eye dominant. My right eye is dominant enough I shoot everything with both eyes open, but it gives me fits when I try to shoot a long gun from my left shoulder, as in carbine transition drills.
Rick