There are a few diferent factors that affect point of impact versus point of aim with a handgun, as the load changes:
1) Pistols recoil to a much more noticeable extent than rifles. In practical terms, due to the light weight of the pistol, the muzzle rises significantly before the bullet ever leaves the barrel. That changes the angle of departure. In turn how much the muzzle rises with one load compared to another affects the point of aim. Generally speaking, lighter and faster = lower.
2) One variable is bullet weight. Most .38 Specials and .357 Magnums, for example, have sights regulated for a 158 gr bullet. If your .357 shoots to point of aim, for example, at 10 yards, and you shoot a 125 gr load instead, you'll find it impacts a couple inches low even at that short range. That's due in part to the higher velocity of the bullet leaving the bore sooner, before the departure angle can increase as much, and due in part to lighter bullets having less recoil than heavier bullets, when everything else is basically equal.
3) With bullets of the same weight, velocity makes a big difference. Higher velocity does mean less time in the barrel, but it also means much higher recoil when the differences are large, such as moving from 1300 fps to 1800 fps. Here the effects are not as clear, as faster equals lower, but there's also an inflection point where significant increases in recoil and more muzzle displacement start to offset and reduce the effects of the faster bullet.
4) The powder charge makes a difference as well. For example you might be using around 17 grains of Win 296 in a .45 colt case to get 1300 fps compared to just north of 37 grains of Win 296 is a .454 case to get 1800 fps with similar bullets. Even if the bullet weight is identical, that extra 20 grains of powder still has mass even after it burns, and unlike the bullet, that mass leaves the barrel at about 3 times the muzzle velocity of the bullet. That more or less equates to a 60 grain increase in bullet weight. Faster burning powders that require less powder produce less recoil for this reason, even when the bullet velocity is the same or slightly higher. You'll see this effect a lot with powders like Unique in some of the smaller magnum handgun rounds.
-----
Regardless of the cause, it'll be what it is. You'll need to back up to somewhere between 10 and 25 yards and zero with that load, then either increase the range in increments to ensure you're still on the paper, or use a ballistic program, in putting the known velocity, and sight height (distance from center of the bore to top of the blade, or center of the scope). If you're guestimating the velocity, you'll still be on the paper but will need to make more adjustment to your final zero.
This approach ill work because the load is the same, and thus the recoil and muzzle rise before the bullet departs is the same, so the differences will once again just be due to the external ballistics.