Do what you want.
Those who think the upper loses accuracy have yet to explain it, tho. If the sights are on the upper - ok Barrel moves, sights move, sight alignment hasn't changed. The hammer to firing pin alignment did, tho. So how does a minute change in the hammer face striking the back of the firing pin change accuracy? That is the ONLY change -
I will tighten up my new AR pistol build for one reason - it's for hunting, and any noise in the field out of the ordinary could spook a deer in close proximity. I did carry a GM Hydramatic that did that in basic, but with all of the other gear and 30 other guys, one rifle making a ticking noise irregularly isn't a big deal. 60 stomping boots kicking thru the forest floor is. With me still hunting slowly, the potential noisemaking ability of a loose fit might ruin an opportunity. Applying that concept goes even further - free floats ringing like little dinner bells every time a branch hits them isn't good either. So, no free float. I made that mistake on my 6.8 - it's a constant hassle stalking in dense cover.
But worrying about a loose fit hampering accuracy? Depends on the barrel first. If it's milspec, it's a 2MOA gun, no grounds to complain. If it's a .5 MOA gun with handloads, whatever. Do consider, tho, that slinging up will pressure that accuwedge and the upper will move in relationship to the lower. It's NOT a metal to metal fit, it's metal on polymer, and exerting pressure means it will deform, and the upper will change relationship to the lower.
That darn hammer is going to move to one side of the firing pin and mess up your group anyway.