The shoulder-to-shoulder method was the best method of fighting with such weapons under battlefield conditions at that time. As has already been mentioned, infantry that did not hold formation was at the mercy of cavalry. Infantry that did not hold formation could not effectively deliver the firepower that their muskets had to offer, either. All the marching and drilling that the military does to this day, comes down to us from that time. The various drills were designed to allow a formation of infantrymen to maneuver as a group as quickly and effectively as possible. Instead of thinking of them as individual men with weapons, think of the formation itself, with its massed fire, as the weapon. The best troops, in those times, were the ones who had the discipline to retain their formations even under fire and successfully close with the enemy. Once an enemy formation had been broken, it ceased to function as an effective fighting unit. One guy with a musket isn't much of threat, militarily. Although we are brought up in this country cherishing the image of the rifle-armed Minuteman sniping at pompous British officers from behind the trees whist the British continue to stupidly stand in a group and get shot down, there was comparatively little of that activity. Did you know that the Brits had marksmen, too, btw? George Washington's great accomplishment was forging an armed rabble into something resembling a disciplined European-style army. Don't underestimate that style of fighting. The British and the French conquered huge territiories using it. Our own army practiced it. Disregard for the common soldier or not, no military knowingly chooses and uses suicidal/ineffectual methods for long. Occasionally, as in the American Civil War and WWI, there are lags when tactics are leap-frogged by weapons; but, the British et. al. used the formations you describe because they were the most effective way to fight using the weapons of the day.