To no one’s surprise, Peyton Manning leads all quarterbacks (QBs) with a 9.7% touchdown rate (TD%) through Week 2. And after ranking third in 2013, Russell Wilson’s 7.5% TD% ranks second so far this season. The rest of the top 10, however, features less credentialed names like Ryan Fitzpatrick (7.3%), Kirk Cousins (6.1%), and Derek Anderson (5.9%). If we had to guess, we’d lean towards Manning and Wilson remaining in the top 10 through January, while Fitzpatrick, Cousins, and Anderson regress to the mean.
The reason? Intuitively, we believe that TD% is nowhere near its stabilization point yet. And based on Monte McNair’s previous work, our intuition is a good guide here: He found that it takes about 1,125 pass attempts, or about two full seasons. Today, I’m going to update/improve his research and report a more valid and precise answer to the question “When does TD% stabilize?”
Same as before, except for a couple of minor tweaks that were necessary due to the stat in question:
- Given the more random nature of TD%, I spread out the intervals for pass attempt groups from increasing 100 at a time to increasing 200 at a time. So, instead of looking at QBs with 200+ attempts, then 300+, then 400+, and so on, I went from 200+ to 400+ to 600+, etc.
- We’re talking TD%, not yards per attempt, so I calculated the “true” ability for a hypothetical QB that’s posted a 5.0 TD% through X number of attempts.
|Attempts||n||r||R2 = 0.50||Avg TD%||Obs 5.0 TD%|
As always, we read the table like this: There were 16 QBs who had (at least) two sets of 1,400 attempts for the same team, and those QBs averaged a 5.0 TD%. Given their split-half correlation of 0.53, TD% stabilized at 1,245 attempts for that group. And given their 5.0 TD% average, we can estimate that a QB with a 5.0 TD% after 1,400 attempts has a “true” TD% of 5.0%.
The “Wtd Average” row is the meat of the matter, telling us that TD% stabilizes at 1,059 attempts. Furthermore, we can use this total along with the league-average TD% to estimate that a QB with a 5.0 TD% after 1,059 attempts has a “true” TD% of 4.8%.1
With our entree out of the way, there’s one side dish I’d like to chew on. Lurking in the results table, there’s an important methodological detail that should not be overlooked. Namely, for the 1,400+ group, it looks like a hypothetical QB with an observed 5.0 TD% also has a “true” 5.0 TD%, perhaps implying that we now know his “true” ability to throw TDs.2 However, in the measurement theory on which reliability analyses like this are based, an individual’s “true” ability is an abstract, unobservable concept, and so we can never actually know it.
This is borne out by the math. Remember, the denominator in our formula for calculating “true” ability is the number of observed attempts plus the number of attempts at which TD% stabilizes. In the case of our hypothetical QB in the 1,400+ group, this equals 2,625 attempts, 47% of which (i.e., 1,245) represent regression to the mean. But notice, no matter how large the number of observed attempts gets, we still have to add the 1,245 league-average attempts. Even at some impossibly large number of observed attempts, say 14 million, we would still only be 99.99% of the way to knowing a QB’s “true” ability.
Furthermore, the results table hints at another, less mathematical, proof of the idea that we’ll never be able to observe “true” ability: dwindling sample sizes as attempts increase. The row associated with the 1800+ group tells us that, since 2002, only eight QBs have thrown more than 3,600 passes for the same team. Tom Brady’s 6,170 leads that group, and Drew Brees is second with 4,990, so a 2,500+ group would have a sample size of one, and any split higher than 3,085 would have a sample size of zero.
Even if I didn’t restrict the sample to 2002-2013, the NFL record for most attempts with a given team is Dan Marino’s 8,358. And if I also removed the “same team” restriction, it’s still the case that no human being has ever observed more than Brett Favre’s 10,169 NFL pass attempts, which means that the maximum split possible is two sets of 5,084.3 Now, consider the following: If we add 1,059 league-average attempts (i.e., the stabilization point I found) to this maximum split, then the totality of NFL history still leaves us 17.3% short of knowing the “true” TD% ability of any QB who ever lived.
DT : IR :: TL : DR
A QB’s TD% stabilizes after 1,059 pass attempts, which is over twice as long as it takes for yards per attempt. In short, there’s much more regression to the mean; so much so that, even after a league-record 10,169 throws, Brett Favre’s 5.0% observed TD% gave us only 82.7% of the information we needed in order to know his”true” touchdown-throwing ability.
Which you’ll recall represents the point exactly halfway between observed performance and league-average performance. ↩
It turns out that the reason the two numbers appear to be equal in the table is simply because I intentionally rounded to one decimal place: The group’s average is actually 5.03%, and the hypothetical QB’s “true” ability estimate is actually 5.01%. ↩
Given the proliferation of passing in the NFL, no doubt someone will break Favre’s record in the intermediate future. For today’s purposes, though, that doesn’t affect the limits of our knowledge as I’m writing this. ↩