The manganese phosphate (a type of Parkerizing) is only used on the steel parts of the rifle (barrel, bolt/carrier, front sight base).  The aluminum upper and lower recievers are made of aluminum, and are hard-coat anodized (HCA), with black dye added to the mix.

The finish can and does wear off eventually (ever see a military M16, which are nearly white from overcleaning?).  However, if you're wiping it down with a light-colored cloth, it will pull fibers out of the cloth and make it look like the finish is coming off.  I use an old black T-shirt for wiping down my black rifles.

-Troy