Your bullets are dropping with respect to your POA,... so your barrel is going limp on 'ya in it's FF tube? It's bowing down after a few shots? Weird!
Try shooting next time not so much from a vise, but put a cork or something in between your FF tube and the bottom of your barrel that won't hurt it, yet keep it relatively in place. At 100yds, you shouldn't notice any real big change, maybe an inch, tops. See if after 15 shots you're relatively in the same hole still.
If you don't feel like doing that, or can't find a cork, actually rest the barrel on the sandbag, not the FF tube.
Regardless of results, maybe Douglas would be willing to maybe take a look at your upper or gauge/lay-out your barrel? Something sounds amiss, obviously. See if you can make heads or tails of it, though, shooting with the actual barrel supported.