It’s been five years since I piggy-backed off of this 2011 Monty McNair post, which itself piggy-backed off this 2011 Tom Tango post, which itself piggy-backed off this 2007 Russell Carleton (aka Pizza Cutter) post, and set out to determine when various NFL stats “stabilize” — meaning the time at which past performance has a 50/50 shot of predicting future performance. Therefore, now seems like as good a time as any to revisit the series. And as before, I’ll start with quarterbacks (QB). For anyone new to the site, my previous analyses found the following:
- Average Depth of Target (aDOT) stabilized at 286 targeted throws.
- Adjusted Net Yards per Attempt (ANYA) stabilized at 326 dropbacks.
- Yards per Attempt (YPA) stabilized at 396 attempts.
- Touchdowns per Attempt (TD%) stabilized at 1,059 attempts.
- Interceptions per Attempt (INT%) stabilized at 1,681 attempts.
In laymen’s terms, these findings meant that knowing a QB’s aDOT, ANYA, or YPA was far more valuable for predicting his future than was knowing his TD% or INT%.
So, armed with five more years-worth of data, have these stabilization points changed in a meaningful way? Read on for the answer(s).
Methods
For anyone new to the site, here’s my standard reliability analysis procedure:
- I collected Pro Football Focus (PFF) game log data from 2006 to 2018.
- I included, and treated separately, all QB tenures for the same team from 2006 to 2018 that lasted at least 8 games. For instance, Ryan Fitzpatrick’s 5 games with the Rams aren’t included, but his 14 games with the Buccaneers are included and analyzed separately from, say, his 14 games with the Bengals. In other words, Buccaneers Fitzpatrick is treated as a different QB than Bengals Fitzpatrick in the analysis (as I think we all can agree he should be).
- Starting with these baseline tenures, I randomly selected two sets of 4 games for each QB, and calculated their efficiency stats in both sets.
- I calculated the split-half correlation (r) between the two randomly-selected sets of games.
- I performed 25 iterations of Steps 3 and 4 so that r converged.
- I repeated Steps 3-5 in 8-game intervals, from 16+ games all the way to 48+ games.
- For each “games played” interval (i.e., 8 through 48), I calculated
- the number of games at which each metric’s variance explained, R2, would mathematically equal 0.5. ((The formula is (Games/2)*[(1-r)/r].))
- the “true” value for a hypothetical QB that’s had an observed performance of 9.00 aDOT, 6.00 ANYA, 7.00 YPA, 5.00 TD%, and 2.50 INT% through that number of games. ((The formula is [(Observed Performance * Games) + (League-Average Performance * Stabilization Point)] / (Observations + Stabilization Point) ))
- I calculated a weighted average of the results from Step 7. ((Weighted by group size.))
Results and Discussion
Below are the updated stability tables for each of our five QB stats of interest. For those that are new to the site, here’s how to read them:
- The “n” column tells you how many QBs were in that row’s analysis.
- The “r” column tells you the statistical relationship between one half of the QBs’ data with the other half (i.e., the split- half correlation). The higher and closer to +1.0, the better.
- The “R2 = 0.50″ column tells you how many games it takes for the relevant stat to stabilize for each row.
- The “Wtd Avg” row tells you the take-home message of the analysis. For instance, in the table below, aDOT takes 10 games to stabilize, the league average is 8.86, and a QB who has a 9.00 aDOT after 10 games with his current team actually has a “true” aDOT of 8.93.
| Games | n | r | R2 = 0.50 | Avg aDOT | Obs 9.00 aDOT |
|---|---|---|---|---|---|
| 4 | 210 | 0.28 | 10 | 8.82 | 8.87 |
| 8 | 114 | 0.59 | 6 | 8.87 | 8.95 |
| 12 | 80 | 0.50 | 12 | 8.88 | 8.94 |
| 16 | 60 | 0.55 | 13 | 8.88 | 8.95 |
| 20 | 47 | 0.64 | 11 | 8.92 | 8.97 |
| 24 | 43 | 0.70 | 10 | 8.89 | 8.97 |
| Wtd Average | 10 | 8.86 | 8.93 |
Given an average of 27.9 targeted throws per game in this sample, a duration of 9.9 games means aDOT takes approximately 276 targeted throws to stabilize.
| Games | n | r | R2 = 0.50 | Avg ANY/A | Obs 6.00 ANY/A |
|---|---|---|---|---|---|
| 4 | 210 | 0.28 | 10 | 5.46 | 5.61 |
| 8 | 114 | 0.36 | 14 | 5.79 | 5.87 |
| 12 | 80 | 0.45 | 15 | 5.95 | 5.97 |
| 16 | 60 | 0.52 | 15 | 6.08 | 6.03 |
| 20 | 47 | 0.57 | 15 | 6.19 | 6.08 |
| 24 | 43 | 0.64 | 14 | 6.21 | 6.08 |
| Wtd Average | 13 | 5.78 | 5.89 |
Given an average of 31.8 dropbacks per game in this sample, a duration of 31.9 games means ANYA takes approximately 407 dropbacks to stabilize.
| Games | n | r | R2 = 0.50 | Avg YPA | Obs 7.00 YPA |
|---|---|---|---|---|---|
| 4 | 210 | 0.24 | 13 | 6.84 | 6.88 |
| 8 | 114 | 0.36 | 14 | 7.01 | 7.01 |
| 12 | 80 | 0.40 | 18 | 7.09 | 7.06 |
| 16 | 60 | 0.46 | 19 | 7.16 | 7.09 |
| 20 | 47 | 0.55 | 16 | 7.21 | 7.10 |
| 24 | 43 | 0.61 | 16 | 7.24 | 7.10 |
| Wtd Average | 15 | 7.01 | 7.01 |
Given an average of 29.9 attempts per game in this sample, a duration of 15.0 games means YPA takes approximately 447 attempts to stabilize.
| Attempts | n | r | R2 = 0.50 | Avg TD% | Obs 5.0 TD% |
|---|---|---|---|---|---|
| 4 | 210 | 0.11 | 32 | 3.9% | 4.0% |
| 8 | 114 | 0.28 | 21 | 4.2% | 4.4% |
| 12 | 80 | 0.28 | 31 | 4.3% | 4.5% |
| 16 | 60 | 0.38 | 26 | 4.4% | 4.6% |
| 20 | 47 | 0.43 | 26 | 4.5% | 4.7% |
| 24 | 43 | 0.47 | 27 | 4.6% | 5.8% |
| Wtd Average | 28 | 4.2% | 4.6% |
Given an average of 29.9 attempts per game in this sample, a duration of 27.9 games means TD% takes approximately 834 attempts to stabilize.
| Games | n | r | R2 = 0.50 | Avg INT% | Obs 2.50 INT% |
|---|---|---|---|---|---|
| 4 | 210 | 0.13 | 26 | 2.94% | 2.88% |
| 8 | 114 | 0.17 | 39 | 2.70% | 2.67% |
| 12 | 80 | 0.19 | 53 | 2.69% | 2.66% |
| 16 | 60 | 0.34 | 31 | 2.62% | 2.58% |
| 20 | 47 | 0.38 | 33 | 2.52% | 2.51% |
| 24 | 43 | 0.35 | 46 | 2.54% | 2.53% |
| Wtd Average | 35 | 2.75% | 2.63% |
Given an average of 29.9 attempts per game in this sample, a duration of 35.3 games means INT% takes approximately 1,054 attempts to stabilize.
Comparing the results above to those from five years ago, it’s clear that, although the details have changed, the view from 30,000 feet remains the same. Yes, TD% and INT% appear to have become “stickier” stats over the past half-decade, but they’re still much less reliable than the other three stats. Therefore, if you’re trying to predict a QB’s future with his current team, you’re still much better off focusing on aDOT, YPA, or ANYA than you are focusing on TD% or INT%.
DT : IR :: TL : DR
Forgive me father for I have sinned. It’s been five years since my last regression (to the mean). My penance is this post, which benefits others by once again demonstrating that Average Depth of Target (aDOT), Adjusted Net Yars per Attempt, (ANYA) and Yards per Attempt (YPA) are (still) the guiding lights of quarterback analytics. To wit, here’s how long they take to represent 50% skill and 50% luck in 2019:
- aDOT = 276 targeted throws
- ANYA = 407 dropbacks
- YPA = 447 attempts