Prefferably, irrespective of the weather, I like using my OTB (on the belt) holster. It is way more comfortable than IWB. However, because of concealability, I generally carry using an IWB holster (in my case, a Comp Tac C.T.A.C.) in the summer. In the winter (when wearing a coat/jacket/parka), I generally carry using my Mitch Rosen OTB holster (a 5JR). Unless it is downright cold, I leave the garment open so that I can access my pistol easily, yet maintain concealment.
As I see it, the fastest draw is when the gun is already in your hand; thus, the issue of situational awareness comes into play.
Yes, there will be a time when you do get surprised and it will not matter what method of carry you are using. However, for those 99 out of 100 times that you can "see trouble coming," get yourself away from the potential danger, OR, get your hand on the weapon (NOT necessarily drawing it from the holster). If you do this, then the type/style of carry is not an issue.