I agree with Tweak. Over time, a properly fitted buffer/pin combo might show a little wear of the finish, but you really shouldn't have any dings. You might also get a smear on the very middle of the buffer face, from it's slight contact w/the rear bottom edge of the carrier as the action is closed, but again, that shouldn't be a gouge.
Interestingly, the buffer face on my 1978 purchased Colt SP1 carbine, with thousands upon thousands of rounds through it, shows less wear on the face than any of the aftermarket pieces at my house. The surface finish looks the same, but apparently is a lot more durable. Ditto for the surplus A1 buffer than came with the stock assy I got from CDNN. That one only has a few hundred rounds at this point,, but the DPMS piece in another rifle has only 100 or so more rounds, and the finish is really worn.
Don't know if the finish on current Colt pieces is the same as they used to use, but the old timer Colt buffers I have sure beat the tar out of the commercial buffers I've seen for durability of finish.