What Homo didn't bother to mention is that your front sight isn't canted to the right. It's tilted LEFT. That's why you're shooting to the right.
Also, keep your files in the drawer until you get the thing apart and take a look at what's really going on. The vast majority of these right shooting situations occur because the barrel was over enthusiastically tightened on assembly, and the locating pin on the barrel dug into the left side wall, pulling the whole front sight assy over with it. If that's the case here, and the odds say that it is, what you need is a shim; not a file.
Take the barrel out, than reinsert it, meking sure that the locating pin is fully bearing against the right wall of the channel. Then start inserting feeler guages in the left side of the channel. My hunch is that you'll find that you can get at least a .0003 guage in there; possibly more. When you get to the thickness that won't fit, go back to the thicknes .0001 less than the last one that did, and snap a chunk of it off in the slot. That'll keep the barrel from rotating bck when you tighten it.
If, in fact, you have no open area to the left, that's the time to go with the file on the RIGHT sidewall, just a thousanth or two at a time. The alignment technique suggested above is pretty crude, but I guess it'll at least get you into the ball park. [:D]
[Edited to add] Why do I think this is a Bushmaster upper? Maybe because they were famous for the overtightening number a couple of years ago. They had apparently hired some assemblers who didn't have a clue, and for awhile we were getting almost daily reports on the site about brand new Bushies shooting to the right.[;D]