A 24" barrel is wasted on shooting paper at 100 yards. Accuracy at that distance is a function of barrel quality and not length. And, no, the two do not necessarily go together. If you want sub MOA groups at benchrest distances, then get a match grade 20" barrel, typically stainless, with a twist rate that is optimal for the weight and length of bullet you will be shooting. Think Kreiger or Lilja as good examples.
Now, having said that, if you ever start to use the rifle for shooting beyond 100 yards, especially for hunting beyond 200 yards, the extra velocity from hand loaded 5.56 bullets and slow burning powder can give you up to 200 fps more velocity and that translates into more retained energy for a clean kill, a flatter trajectory and for really long shooting, an extension of the distance as which the bullet goes transonic and possibly destabilize adversely affecting accuracy. That is not likely until you approach perhaps 600 yards, however.