I think that the logic of not trimming pistol brass comes clearer if you compare straight wall pistol cases to bottleneck rifle cases.
Sizing both types of cartridges is necessary to restore the case diameter so that it will chamber correctly the next time, but bottleneck cases are more aggressively worked while sizing because they require that the shoulder be set back to pre-fired condition.
Then when a bottleneck cartridge is fired the next time, the shoulder is pushed forward again to meet the shoulder of the barrel chamber.
It's because of the more aggressive sizing of bottleneck cartridges required by their shoulders stretching forward (which takes the case mouth forward with it) that they "flow" forward (longer) each time they are sized and have to be trimmed often, but since there is no shoulder on a straighwall pistol case, they aren't worked so much when fired and only require sizing back to their proper chambering diameter.
Pistol cases do stretch over time, but very little, and I have found what others have mentioned that the mouth of a straightwall case will virtually always split and cause it to be tossed away much earlier than case stretching would be an issue.
I mic a few in a batch once in a while, but have never come even close to needing trimming.
For the record, all of my pistol reloading consists of .357, .38 Special, some .44 Mag, and a plethora of 9mm, and I do mean plethora.