I think there is a middle ground to be had on this subject.
For example. You don't really need 2 rounds that are very close performance wise in the same "form factor" of handgun (eg... 9 and 40).
It makes perfect sense to have 2-3 auto pistol calibers sized proprortionaly to the gun. Example .380 mouse-subcompact, 9mm compact-full, 10mm full size.
For defense, if you are going to stick to 1 round. Then stick to one gun type too so mags interchange. Like say GLOCK 17 for HD and GLOCK 26 for CCW.
Also. Rounds vary on availability in a panic. Consider a couple of obscure or east bloc rounds. During the 08 and 13 panics. 9x18 was still relatively available when 9 and .380 were scarce. Also, a caliber that isn't associated with high caps (.45, revolver rounds) tends to stay available.
With 4 calibers, keep 2000 rounds for main, 1000 for the rest. That's only 5k and will get you through most panics.