Well, I'm not trying to be harsh, but here's my opinion.
If you said you would pay shipping, then the responsibility falls on you to ensure that he receives the package in the advertised condition and in a reasonable amount of time.
Having said that, this is a perfect demonstration of why shipping terms should be clearly delineated when discussing the terms of the deal. It may have cost you a couple of bucks to upgrade the shipping from 1st class to Priority and add $100 of insurance. I'm sure that right now, you will agree it would have been a couple of bucks well spent.
I have to agree that it is not unreasonable for the buyer to request a refund at this point, but I would encourage him to wait a few days more to see if the package arrives. Your post here demonstrates to me that you are genuinely concerned that the buyer is satisfied, and that you're probably not going to flake out on him if the package ends up being lost.
Good luck and hope it works out for both of you.