Just a suggestion...
Make sure your slide is closing all the way. If it's not, that could be absorbing some of the hammer's energy the first try. And, it could be a warning sign for a very dangerous situation.
A CZ-52 I used to have would not close reliably on whatever ammo it was I bought a case of. I determined that this was due to the chamber neck being too tight. This may have been due to the primitive crimp the ammo had distorting the case neck, or it may have been that the chamber was actually cut too tight. The latter situation is very dangerous -- there needs to be enough clearance for the case neck to expand and release the bullet, or chamber pressures can skyrocket.
In my case, I recut the chamber neck area with a .30 Carbine chamber reamer I have, which made the pistol function perfectly and allowed my blood pressure to drop back to normal.
The ammo I was using had 3 or 4 "centerpunch marks" for a crimp.
While this pistol's locking system is ideal from an accuracy standpoint (the barrel's movement involves no rocking or tipping) I do not personally think it's as strong as most people seem to believe. I also hate the decocker, and feel the firing pin lock is poorly designed and not to be relied on. And my pistol also had too long a firing pin, allowing the firing pin tip to protrude from the breech face with the lock engaged.