I recently got a surplus hi power ('76 C prefix) and love it. I don't really mind how small the vintage hi power safety is. Only major issue is that the safety is very mushy and sort of feels like it drags and does not click. I have had the gun apart and the detent in the safety and in the frame look fine. I oiled both and removed the grime hoping for a change, but it didn't work. I wanted to replace the spring but cannot get the tiny pin out of the safety. Does it remove from top to bottom? I buggerred the pin head a little trying to get it out that way and am now thinking it may be the opposite way?

The crazy thing is that the WW2 German occupation made hi power that I inherited from my great uncle has an awesome feeling safety. It clicks positively and feels excellent.

Any advice would be appreciated.

Unfortunately the design of the Hi Power's thumb safety doesn't lend itself to the crisp feel you may know from a 1911 and will always be a bit mushy. Just like the trigger will never be great either, but both can be cleaned up some.

The older thumb safeties did have a small retaining pin. IIRC they were roll pins and could go in/out either direction. You might need to secure the safety in a vice to move it. But I expect the fit of the safety is the ultimate problem you're facing.

A mkIII or C&S safety will feel more positive.

Just about everything on the HiPower is mushy.  Nature of the beast.  Browning had to design around his earlier patents as Colt owned those.

Once you get the pin out, replace the spring with a Wolf spring as it applies more tension and while it's out take a dremel with a tiny ball grinder type tip and slightly deepen the divots that the safety plunger engages.  Use some kroil and heat on the pin prior to whacking it.

I always assumed that it was related to the safety spring and detent. I just replaced mine with a C&S safety and in fitting it, learned a lot about how the safety works. Fitting the new one, made mine pretty crisp but it took a couple of hours of dis and reassembly.