Just about any height of optic will work, its just a matter of comfort and preference.
Both the wire and folding stock sit a bit lower on an M11 than the top rail so you really dont need to mount a optic higher up like you do on a AR15 which has the top o stock basically level with the top rail.
Keep in mind there isnt much recoil so getting a super solid cheech weld isnt all that necessary.
Video of a Aimpoint T series optic with low profiel mount. The stock is higher up on the shooters cheek
MAXimizing the M-11/9 Sub-Machinegun
Video of a full sized optic at AR15 height. The stock is lower on the shooters cheek
MAC-11 Shockwave ZMAG Review - FULL AUTO
Since my M11 is just a range toy I went with a Holosun HS503GL, but its basically a Aimpoint T1/2 clone.
The only low profile iron sights I know that for sure work with a low profile optic are those sold by lage. Honestly, I'm not a big fan of them, they are not of the same quality as most lage products.
Mine was crudely milled with a rough finish, it didn't adjust well and the rear lines to show adjustment is a plate screwed on, not lines etched into the sight....this plate can easily move around. The front blade is rather wide and the rear hoop is IMO larger than it needs to be, this combined with the fact that is co-witnessed it basically covers up most of my optic since its only a 20mm window.
Attached FileOne option you might want to look into is HK MP7 folding sights. When down they act as a low profile pistol sight, but can be flip up for better accuracy. With that said I don't know the heights or how it would work with a Aimpoint T type optic. At $300+ for the set I'm hesitant to try, but might pick up a airsoft knock off just to test dimensions.