After getting all the gunk out of a friend's XD45's striker channel so it would fire, I ran a couple boxes of ball through it.  

Probably half a doxen FTFs where the round would nosedive back into the magazine rather than feed normally.  

I've not seen this failure mode before.  Any ideas on what causes it and how to fix the problem?

Is it sensitive to what magazine you're using? Only have one or two? If so, it could be the magazine springs If memory serves me, when I got my XD45, there was some talk about the magazine springs being weak. I think I replaced all mine with aftermarket springs that were a tiny bit stronger. I've not had feeding problems except with a couple varieties of HP ammo early on (before I replaced the springs, I think). It feeds ball and HST fine now.

I'm going with a mag issue.  Prolly a worn spring, and it is a good thing to pull the basepad off and clean it out every 500 rds.

+1
Although, how many rounds on the recoil spring?
Replacing both couldn't hurt.