To anyone who is interested, I confirmed that my Carbon 15 upper receiver was to blame for the accuracy problems. I tried different bullets, scopes, bases, a free float hand guard, and putting jb weld between the picatiiny rail and the upper. Even after that, I could grab the barrel, and grab the scope with separate hands, and torque them in different directions fairly easily.
To fix the problem I bought a new solid aluminum upper receiver, and kept all my old parts (barrel, scope, mounts, hand guard, ect.). Right after I swapped out uppers I noticed I couldn't torque the scope and barrel in opposite directions. I took the gun to the the range and it went from a 7 MOA to a 1 MOA (8 shot group). For a Bushmaster M4, this is more like it! To top it off, I was only using cheap re-manufactured ammo.
The new upper doesn't match with the Carbon 15 lower perfectly, but most people won't notice unless I point it out. Nonetheless, problem solved!
If anyone else it having major accuracy problems with their Carbon 15, grab the barrel and scope to see if they torque in opposite directions. Even a slight noticeable move can equal several inches at a 100 yards.