A few weeks back, SinistralRifleman inquired about my new Mossberg 930 SPX. The conversation was as follows:


The gauntlet had been thrown down. I was determined to test the shotgun. On hearing of the impending test, my buddy Absolut_Zero decided to get in on the action with his brand new FNH SLP Mk. I. And the Autoloader Grudge Match was born.

This grudge match is brought to you by the Awful Shooting Squad, your friendly bunch of idiots who do shooting competitions.

http://blog.nickleghorn.com/wp-content/uploads/2010/12/teaser-272x300.jpg

Methodology

The basic outline of the test was simple: two shotguns, 500 rounds and 500 clay pigeons each, most failures loses.

In order to properly test the shotguns a lot of variables had to be controlled for.

First, two ammunition types were purchased for the test, Remington #8 shot target loads and Winchester Super-X #8 shot. In order to keep the test fair each shotgun would fire 250 rounds of each ammunition, alternating ammunition types with every new box (25 shells), and both shotguns firing the same ammunition at the same time.

Another issue was the skill level of the shooter. 6 shooters total participated in reaching the 500 round goal, shooters rotating every box of ammunition and switching shotguns every time they rotated in to make sure that any limp wristing of shotguns or other user error due to inexperienced shooter's issues would be evenly distributed among the data set. In order to test that even distribution the average number of hits per box of ammunition was chosen as the metric, with the assumption that novice shooters would do worse and experienced shooters better, so an equal average would indicate that the shooters were not an issue in the data set.

In addition to the quantitative testing, a bit of qualitative testing was conducted as well. A test was conducted to determine if the "smoothness" of the action degraded after 500 rounds based on a 1-6 scale as determined by the participants. Additionally, a survey was circulated following the testing to see which shotgun was preferred for an array of factors.

Results and Analysis

All results were based on a box of ammunition (25 rounds) being the unit of measure. Both shotguns successfully fired 500 rounds of ammunition within a 4 hour time frame.

The first result computed was the average hits per box of ammunition to see if the firing of the shotguns had been evenly distributed between the various skill levels present. The following boxplot indicates that the average was precisely the same and the distributions almost identical, meaning that the results would not be biased by the skill levels of the shooters involved (assuming more experienced shooters are more accurate).

http://blog.nickleghorn.com/wp-content/uploads/2010/12/hits-300x300.png

A Welch two sample T-Test confirmed that the two distributions were statistically similar. The T statistic was 0.144 (which, with 37.83 degrees of freedom, indicates that any differences in means is probably a chance finding), the P-Value in this test was 0.8863, and the 95% confidence interval was closely straddling zero (-2.612411, 3.012411).

Because the experience level of the shooters was as controlled for as possible within this experiment, the only variables left were the reliability of the shotguns and the ammunition types in use. The next result calculated compared the average number of failures per box of ammunition for both shotguns.

http://blog.nickleghorn.com/wp-content/uploads/2010/12/failures-300x300.png

This boxplot indicates that while the 930 SPX had a couple of failures they were not statistically significant, and so therefore the average number of failures to be expected per box is 0. On the other hand, the FNH SLP's boxplot indicates that a shooter can expect an average of 5 failures per box of ammunition. On the surface this might seem like a significant conclusion, but a deeper look into the data reveals an interesting anomaly.

http://blog.nickleghorn.com/wp-content/uploads/2010/12/SPX-Malfunctions-per-Box-300x300.png
http://blog.nickleghorn.com/wp-content/uploads/2010/12/SLP-Malfunctions-per-Box-300x300.png

These bar graphs show the total number of failures per box of ammunition over time, with the first shots fired on the left and the last of the 500 rounds on the right. With the 930 SPX, it seems that the failures are somewhat evenly distributed, meaning the issue was not with ammunition types or specific shooters but simply a random malfunction. With the SLP, however, the final 10 boxes show a remarkable pattern. Excluding all but the last 10 boxes it would seem that the SLP simply does not like what turned out to be the Remington target loads. Of these failures the vast majority were extraction/ejection issues, indicating that the load might have been too light for the gun, despite the "light load" gas system being installed on the gun (the SLP has a gas system which must be changed to use lighter loads, the 930 SPX does not). The first 10 boxes show a more uniform distribution of failures, which may be attributed to the SLP's "break-in period" which some shooters claim takes a "long time".

Conclusion
From these results we can determine that, using the average target loads available at convenience stores, the Mossberg 930 SPX is the superior shotgun. When using a "proper" shotgun ammunition in the FNH SLP Mk. I, however, the performance is so similar as to be statistically insignificant after the approximately 250 round break-in period. Despite that seeming parity in performance, the fact that the Mossberg 930 SPX ran every ammunition type we could throw at it with ease and produced equivalent results to the FNH SLP Mk. I without needing a break-in period we can determine that the 930 SPX (at $400 less) is superior to the SLP Mk. I.

As for the qualitative results, the results were decidedly in favor of the Mossberg 930 SPX as well. While the actions degraded their "smoothness" equally, almost everyone surveyed indicated that the sights, the weight, the balance, and the overall function of the Mossberg 930 SPX was superior to the FNH SLP Mk. I.

Full formal results in report format are available here.

Great stuff, thanks for taking the time to do this. I love my 930SPX.

Great writeup.  I just purchased my 930 SPX a few weeks ago after a long eval process against the FN SLP.  I've very happy with my purchase, and I have yet to encounter a single malfunction.

Strange.  I have two FN SLP Mark 1s.  I have never had one malfunction EVER with either.  Hmmm.

Bad links.....

What piston ring was in the FN? If you're using the heavier ring then issues with the birdshot. I own a FN SLP and I have the 1oz or less ring in mine. I can shoot 1oz slug to #8 birdshot without a single malfunction. The heavier gas ring is used for 3 inch shells.



If I remember correctly, FN ships the SLP and Mark I with the heavier gas ring installed. So if you didn't swap out the gas rings then you'll have the malfunctions.



Also once again..... you links are bad.



 

Good test with some detailed data points thrown into the mix. Having said that, I've had the exact opposite results.  I shoot a lot of 3-gun and ditched the 930 for the FN SLP Mk I for reliability and slug accuracy.

Yep.  Something is seriously flawed in this experiment.  The graph shows failure rates for the FN SLP that are absurd.

Links are fixed, thanks for the heads up.

The light load gas ring was indeed in use on the FNH SLP during the test. We made sure that everything possible had been done to the two shotguns in use to ensure reliability including proper parts usage and adjustment, a thorough and detailed cleaning the night before, and using only shooters who had competed in 3-gun competitions. We gave these shotguns the best run possible.

The flaw in this test is that only a single example of each shotgun was used. It's entirely possible that my friend simply bought a "bad" SLP Mk. I and I bought an abnormally "good" 930 SPX. However, if you follow that argument that means that FNH allowed a faulty shotgun to be sold to the public, which opens the door for the possibility that their QC is poor. I prefer to think that, as examined in the full report, the SLP was just picky about ammunition and ended it's break-in period halfway through the test.

This reads as if you're trying to be overly scientific in an effort to conceal your bias.  





The skill level of each shooter should have no impact on reliability.  You are testing for function here, right?






Number of hits made by each shotgun is irrelevant






I don't understand why you chose to include several shooters to grade on
smoothness, and then exclude your findings on that from this report.
Would it have made more sense to have each gun's owner fire all the rounds?  That way the likelihood of operator error is minimized.  By having six shooters participate in this test, you have increased the likelihood of operator error 6x.






You are reporting that people can expect a 20% failure rate from the FN SLP.  That is hard to stomach.  If FN is sending out guns that can be expected to fail 20% of the time something IS terribly wrong with their QC.  I am inclined to believe that something else is going on here. It would seem that this test was doctored to achieve these results by choosing to use the improper gas ring in the FN SLP.



ETA - Just read your previous post, let this show that gas rings were apparently NOT the issue.

fantastic post.

The test is bogus.  The FN SLP is one of the most dependable semi auto guns being manufactured.  You probably used the wrong piston and don't want to admit it.  Or, the gun was not cleaned properly or put back together wrong.  Or, the test never happened and you lied.  

The bar graph shows malfunctions for every single round in a box at one point.  Not possible.  See above explanation.

Number of hits made by each shotgun over the course of the test is, as you correctly identify, rather irrelevant to the final question. However, one of the assumptions we made was that if the distribution of hits/box for each shotgun were similar then the experience level of the shooters was controlled for.

We rotated the shooters because that was the only way to ensure that the average competence of the operator was controlled for. Having the respective owners fire all 500 rounds would be roughly equivalent to having two different racecar drivers run two competing cars around a track - you could never be quite sure if it was the car or the driver that was faster. We made the decision to take the shooter out of the equation by inserting an "average" shooter using 6 rotating shooters. I admit this may be a flaw in the study, but we thought it would be better to see how an average shooter would do with each gun rather than an expert.

In terms of bias towards one gun, I can't see any. The final results were analyzed by both myself (owning the 930 SPX) and my friend (owner of the SLP Mk. I). Both owners were given the opportunity to do everything in their power to prepare the guns, and ammunition was inspected and agreed upon before the testing began. In fact, the ammunition in use was specifically selected because it was the heaviest #8 shot we could find, and #8 shot is the largest shot allowed at the competitions we participate in.



Smoothness was something we investigated, and it did not turn out to be a differentiating factor. As stated in the full report both actions degraded equally, and that degradation over 500 rounds was not serious or unexpected enough to warrant a note.

As stated in the report, these were new shotguns. Also as stated in the full report, the SLP is considered to have a rather long break-in period. It is conceivable that the SLP will improve in reliability once it is fully broken in. As seen in the failures breakout the failure rate drops drastically the longer the test is run which supports this notion. So, after understanding all of the caveats, assumptions and limitations, all this report is saying is that a new SLP using this specific ammunition can expect an average of 20% failure rate within the first 500 rounds. Extrapolation of these results beyond the first 500 rounds would be inadvisable.



We kept count of every failure, including failures to feed, fire, extract and eject. The SLP had multiple instances where more than one malfunction took place.

Another guy with an slp that simply cant believe this stuff. I have never had an issue with my mk1. Im guessing you guys might have gotten the pistons mixed up, its easy to do.

500 rounds. Is that all??? That's not even a good Sunday afternoon at the skeet field. Come see me after 5,000 rounds.

Accountant