I had the same thought when I got into an AR pistol in 223. Faster powders to reduce flash and blast. Problem with the faster powders is reliable function.
If your in a state that allows suppressors, that would be more rewarding than experimenting to optimize a 223 round in a short barrel. The end result will be a minimal payback on your expenditure of time.
Might think about a 300blk since the design intent was to function with a shorter barrel. Just a thought. I went that direction and found the heavier bullet allows lots of load variation is possible if you open up the gas port and use an adjustable gas block as you dial in the load.
For instance, I was able to get reliable function with a 110gr bullet at subsonic velocities from a 7.5" barrel. Recoil was like a bb gun. Fun stuff.
On the frangible bullets... I'd use those for practice on steel plates at close quarters, but not for a defensive round. Better options are available.