Through Week 2, 12 “qualifying” quarterbacks (QBs) have an interception rate (INT%) of 0.00%. Five of them are named Derek Anderson, Kirk Cousins, Ryan Fitzpatrick, Brian Hoyer, and Drew Stanton. Lest you think this an idiosyncrasy of 2014, last season’s top 5 included Sam Bradford ranked fourth, Alex Smith ranked third, and Josh McCown — Josh friggin’ Mccown!!! — ranked first. To put it mildly, INT% seems like a measure that has a ton of inherent error, and therefore isn’t all that reliable as an indicator of “interception avoidance ability.” Indeed, Monte McNair previously found that it takes around 2,250 attempts to stabilize, which was over twice as long as any other QB stat he evaluated.
That said, because of this larger project I’m working on, let’s go ahead and answer the question, “When does INT% stabilize?”
Same shit, different day:
- Because INT% seems even more random than touchdown rate, I spread out the intervals for pass attempt groups so that they increased 250 at a time.
- I calculated the “true” ability for a hypothetical QB that’s posted a 2.5 INT% through X number of attempts.
Here’s the familiar results table:1
|Attempts||n||r||R2 = 0.50||Avg INT%||Obs 2.50 INT%|
Answering our headline question, INT% stabilizes at 1,681 attempts, which means that, if we want to know a QB’s “true” INT%, we have to add 1,681 attempts of league-average performance to the performance we’ve already seen. As an example, Brian Hoyer currently has 0 INTs in 71 attempts. The league average INT% is 2.23%, so we add 37.5 INTs over 1,681 attempts to Hoyer’s stats through Week 2.2 Therefore, the final result for our estimate of Hoyer’s “true” interception-avoiding ability is 2.14%.3
Funny thing about that: If we do the above calculation for every QB through Week 2, we find that the difference between the league leader in “true” INT%, Tom Brady, and cellar-dwelling Matt Cassel, is only 0.25%. Compared to the range of actual (i.e., observed) INT%s, 7.69%, that’s downright microscopic, and gives an indication of just how much regression to the mean plays a factor in INT%. Running the numbers, with the average QB having 64 attempts right now, observed INT% currently accounts for less than 3% of “true” INT%.
This is the first time my results diverge considerably from Monte McNair’s, so it’s worth exploring potential explanations for why my stabilization point is about 600 attempts earlier than his. There seem to be two prime suspects: (1) differences in the data we used, and (2) randomness.
Regarding the former, 2013 is the last year in my data set, whereas McNair’s is 2009. It just so happens that the leaguewide average for pass attempts per game began skyrocketing in 2009 alongside increasing completion percentage and decreasing INT%. Therefore, it could be the case that my data has higher QB accuracy over far more observations (i.e., attempts), both of which tend towards more stability.
Another way my data is probably more stable than McNair’s is that I’m calculating INT% based on a QB’s total attempts for the same team. Ignoring statistics for a moment, our eyes and our intuition tell us that playing in the same offense over time is an inherently more stable situation than jumping from one offense to another. Because McNair’s data doesn’t account for this, it’s likely to be more noisy than mine, which does account for it.
Speaking of noise, the other likely culprit for why our results differ is the statistical variation associated with random sampling and iteration. You’ll recall that McNair and I both calculated correlations for 25 random samples in each attempts group. In my analysis, which evaluated nine groups, that amounted to 225 random samples. But considering the fact that, in some groups, we’re talking about well over 100,000 total attempts, the number of possible random samples is mindbogglingly large. With that in mind, McNair and I probably just covered different areas of the sample space.
DT : IR :: TL : DR
A QB’s INT% stabilizes after 1,681 pass attempts, which is about five times longer than it takes for yards per attempt. The fact that INT% is less reliable than other QB stats isn’t surprising. What is mildly surprising, however, is that my stabilization point is 600 attempts faster than what’s been found in previous research. This is likely due to (a) my data set being inherently more stable, and (b) random sampling variation.