Does anybody know a good way to clean the muzzle brake.

The q-tips.

This stuff is caked on pretty good I can't even get it off with a tooth brush.  I guess I can just scrape it off with my knife.

Try spraying it with some brake cleaner first.  A lot of people use brake cleaner on their guns, me included.  I have even used oven cleaner on the bolt carrier to remove stubborn carbon deposits. I've never harmed a finish,metal or plastic, with this stuff

I thread my boresnake backwards in the side holes and then use both hands to pull the snake back and forth in a buffing like motion.  (I have a Bushmaster Y-comp brake with big side holes.)  I don't let the bronze brush area of the boresnake contact the brake.  I then use a q-tip and put some Tetra grease on each hole of the brake.  This way, the fouling comes right off.  (I've also heard from some IPSC guys PAM non-stick spray works good.)