to control headspace, you ABSOLUTELY NEED the cross pin!!! do NOT use set screws! this is a critical safety feature that, if screwed up, can cause serious injury or death.
as far as a fit goes, yours sounds loose. the commies use .003" to .006" INTERFERANCE fits, thus mandating the use of some sort of press or heat/cold fitting.
this is not to say that you have a setup that is too loose. did you bbl. and trunnion come from a matched kit/rifle that was once assembled to each other? i have heard of guys that turned their bbl. journal down too far and had a slip fit in the trunnion and their rifles are holding up...but i would procede very carefully at this point.
although the cross pin controls the majority of forces that affects headspace, the fit in the trunnion helps control front-to-rear bbl. movement AND accuracy by controlling the side-to-side and up-and-down movement as the rifle goes thru temperature cycling.