It is silver soldered on.
There should be two small dots in front of the sight to show the solder points. It is not going anywhere unless you milled it off. You can try and torch it off, but you will disfigure the slide, which will require a new blue job.
I had a Portugese assembled BHP and absolutely hated the Millet rear sight for the same reason as you......I bought a Browning adjustable rear sight......the cheapy looking sheet metal one and put it on my slide and I was happy.......it was lower and I kept the front sight.....which is thicker and seems to help with accuracy. It cost me $26 total.
I only shoot one kind of 9mm ammo......free US Navy 147 grain JHP and it is zeroed for it.
the first time I went to the range to sight it in......10 people came up to me to buy it!!
Seems "Serpico" just played on TNT and my BHP looked like his.......I saw the flick and it was true......
I liked it even more.
Hope it helps.