Yeah, I'm not sure that going to lighter ammo will solve the problem. The problem would be your rifle itself, something's wrong there, that's for sure.
However, I have found the Ultramax round loaded with the Nosler 40gr ballistic tip to be the lightest load in the market. I have chrono'ed that load right at 3000 fps (20" barrel), which for a 40-gr load is real mild.