In my experience, these are the keys:
1. Pants that fit correctly. They need to be sized to be flat at your waist, but not too tight.
2. Inner belt that's not a soft flimsy POS. It should have at least a little form to it.
3. Size the outer belt to close tight against the inner belt. It should all but "bunch" the inner belt when you secure it. That's how you know it's sized correctly. The inner is pushing out against the outer. The outer is holding everything together like a tight belt sandwich, and the inner is just tight enough to hold your pants up. The belt keepers just hang on for the ride, and keep stuff in place when it's go time.
You probably have the above stuff all good-to-go, so I'd say cut the tabs/buckles off and try adjusting the keepers and holster to be in a slightly different location. I don't pad anything. I used to have that hip-rubbing problem. After I figured out the proper sizing for everything, it went away.