Assuming that the BUIS is holding rock solid on the upper, and is not loosing it zero,
Pull the barrel and confirm that the FSB is in the same index line as the receiver extension pin.
If the two are correctly indexed together, then reinstall install the barrel with it torque to no less than 35 lbs and no greater than 80 Lbs.
Using a barrel vise block, lock the bore to find a distance focal point. Now look down the sights and confirm that the sights centered zeroed and the bore is aiming at the same point down range.
Now using a Leather raw hide mallet , pull the upper out of the vise, hold only the upper receiver and make the FSB blow at the base to barrel to drift the entire barrel in the upper barrel socket to index the FSB into the correct plain.
Now once the FSB is in the correct index, put the barrel back in the barrel vise block and mark the location of the FSB to the barrel with a pencil. Using a rubber mallet, make a few light strikes to the tower once way, then the other. Confirm that the FSB is holding tight to the barrel, and that you have not penned the taper FSB pins, which are allowing the FSB to shift index on the barrel.
Note: My guess is that the barrel nut was not torque correctly from the start. The bolt locking into and disengaging from the receiver extension was causing the barrel to shift index in the upper barrel socket. The fact that the rubber mallet will not drift the barrel when it is correctly torque is normal. Too much of the impact is lost on the cushioning effect of the rubber.