Is it just me but for just normal navigating or scanning I seem to do better w/o the IR source. If I need to really focus on a target I may use it, or looking into a dark ravine, hole etc I get a better overall picture of my surroundings without the light.

I try to practice IR discipline.

And, yes, you're right, if you have too much light thrown out, darker spots will lose contrast and the I² tube will either wash out or lose sensitivity.

Seems to darken the peripheral up, so I keep it off, and if I need to identify something  particular in the distance then the weapon mounted IR comes on. Otherwise just for navigation, it stays off.

It is purely situationally dependent. In many cases vision can be degraded or even obscured altogether because a tube was overloaded by supplemental IR. In extremely dark environments they are extremely necessary. Sometimes supplemental IR improves contrast and sometimes supplemental IR degrades contrast. If you need it you use it, if you don't need it you don't use it. No matter what, light discipline rules always apply as the situation requires - even with IR.

Variable IR light source or variable ND filter will allow the adjustment of IR light to fit the situation.

For inside buildings very bright full power IR light works great, just use it as a bounce light source.
If helmet mounted point it up with a flash kill attached.
The source of the IR light will not be seen pointed up, but reflected light will light things up.
No red/purple glow will be seen moving around.