I'm getting a custom muzzle device made for a factory threaded barrel to be used with a suppressor.  The "factory" barrel thread is supposed to be 9/16x24RH. (It's aRuger AC556).
9/16=0.5625 inches
When I check actual diameter with caliper, I get 0.560 inches.  

I'm not a machinist and my caliper has not been calibrated. Is this acceptable deviation?  (I plan to check dimensions with another caliper this weekend.)

Also, the threaded portion of barrel is 1/2" long and there are 12 threads.    Is the "24" in the 9/16x24 "threads per inch" or "turns per inch"?   It takes 8 turns to screw on the existing muzzle device, so it seems it would be "threads per inch" in this case, but is that the standard meaning of "tpi" in the barrel threading community?

I just want to be sure I'm relaying the correct info to the folks making the muzzle device.

.0025" is well within the tolerance of the OD of a thread.
for reference your hair and a piece of paper is about .003"
TPI is threads per inch
I am a journeyman machinist

Joe is correct.
For your reference:
The minimum major diameter, for a 9/16-24 2A UNEF screw thread, is 0.5541".

Id let the person making the parts mic the threads himself. Actually having the barrel, during the threading process, would be the appropriate way to go about it.

this

The folks doing this are out of state and this is an NFA item.  Do you think sending the existing (removed) flash hider with threads would be beneficial?  That is the thread pattern being duplicated.

You are having someone makes female threads (B).
You want to give them a "gage" so they know when the threads are correct. That would be a set of male threads (A).
I would send the bbl, for 2 reasons.
1.They can accurately cut the female threads.
2.They can determine whether your bbl threads are concentric w/ the bore and if your shoulder is square.

If you are trying to copy certain features of the existing muzzle device, send that along too.
Your bbl is not an NFA item and they should be a licensed (tax paid) shop so this isnt a worry.

Using a mic that way to measure threads is not accurate.

an "ideal" thread has a sharp point and a sharp valley, in the real world it is next to impossible to cut them that way.

Threads are measured with special thread gauges or across "wires"

Read here

I agree, to both check the allignment and concentricity of the threads and shoulder, and to get your "custom" part correct, I would send the barrel.

If you have to send the entire weapon it would be ok, as you should be using a licensed gunsmith anyway, you would just need some paperwork.

Read here

The only thing that would not be ok is if you were sending a title 1 firearm and the resulting work returned a title 2 firearm.

As everyone else has stated, your threads should be in spec. Your caliper will only be able to measure the major diameter of a male thread or the minor of a female thread. The spec will call the actual numbers larger or smaller depending on male or female threads, because you need clearance for the threads to be able to interface with each other. The amount of clearance is specified by the class of fit. Also, just for a reference, threads per inch would be the same as turns per inch.