Similar to the pattern I cited with respect to quarterbacks (QB), Pro Football Focus’s (PFF) average depth of target (aDOT) rankings for running backs (RB) were mostly consistent from 2015 to 2016. For example, among 52 RBs that played at least 25 percent of team snaps in both seasons, David Johnson ranked second in 2015 (4.6 aDOT) and second in 2016 (4.6), while James Starks ranked 46th in 2015 (-1.4) and 51st in 2016 (-1.8). However, unlike at QB, there also were glaring instances of inconsistency. For example, Jerick McKinnon dropped from 15th (1.5) to 36th (-0.1), LeSean McCoy dropped from 18th (1.3) to 48th (-0.6), and Spencer Ware ascended from 47th (-1.7) to 5th (2.7).
Of course, you could argue that all three of these inconsistencies can be explained by differing offensive circumstances between seasons (e.g., McCoy’s 8 games without Watkins) and how defenses adjusted to those changing circumstances. And you’d probably be right; but therein lies the rub. You’ll recall that I prefer my analyses to focus on player games/seasons with the same team. McKinnon, McCoy, and Ware, although having played on the same team across both seasons, turn out to be examples of why. To wit, in this sample of 52 RBs, only 5 changed teams. Here are the changes to their aDOT rankings from 2015 to 2016:
- Matt Forte moved from the Bears to the Jets. His aDOT ranking dropped from 14th (1.6) to 45th (-0.4).
- John Kuhn moved from the Packers to the Saints. His aDOT ranking rose from 37th (-0.4) to 15th (1.3).
- Lamar Miller moved from the Dolphins to the Texans. His aDOT ranking rose from 40th (-0.7) to 23rd (0.7).
- DeMarco Murray moved from the Eagles to the Titans. His aDOT ranking rose from 45th (-1.2) to 29th (0.4).
- Chris Ivory moved from the Jets to the Jaguars. His aDOT ranking rose from 50th (-2.5) to 29th (0.4).
So all five RBs in the “different team” group exhibit aDOT inconsistency, while only a handful of the 47 RBs in the “same team” group do. Got it, but I (methodologically) digress.
In light of the above, today’s reliability seeks to answer the question, “How many games/targets does it take for a RB’s aDOT to stabilize?”
I’m trying to save space for more-than-the-usual commentary, so click here for details on the procedure I used for QB split-half reliability analysis, which applies to the current RB analysis as well.
Below is the usual stability table, this time with respect to aDOT for RBs. Once again, if you’re unfamiliar with how to read this table, click here.
|Games||n||r||R2 = 0.50||Avg aDOT||Obs 1.00 aDOT|
Focusing as always on the “Wtd Average” row, it turns out that RB aDOT takes 14 games to stabilize, i.e., represent 50 percent skill vs. 50 percent luck. Across my entire sample, the average number of targets per game was 2.16, which means 14 games translates to approximately 30 targets.1 In addition, the “Wtd Average” row also dictates that a hypothetical RB with a 1.00 aDOT after 14 games has a “true” aDOT of 0.80 — which you’ll notice is exactly halfway between 1.00 and the weighted league average of 0.59. ((with intentional rounding, of course))
Headlines aside, there are a few subtle revelations in the table that might fly under the radar, so let me bring them to your attention via a somewhat theoretical discussion.
You’ll recall that it takes 10 games for QB aDOT to stabilize, which means RB aDOT takes longer. But of course, you (or I) probably could have guessed that before the analysis; simply via logic. Although “random vs. non-random” (aka “skill” vs. “luck”) isn’t precisely equivalent to “controllable vs. uncontrollable” (e.g., avoiding an interception is controllable, but also relies on a modicum of bad luck), it’s useful to use this “skill equals control” way of thinking as a theoretical starting point. So, when a QB receives the snap on a passing play, who is in more control of the target and its depth? The QB or the RB? Clearly, it’s the QB. Now, as will be revealed in future posts, this may not necessarily be the case for wide receivers and/or tight ends,2 but, as it relates to RBs, lack of control over targets (and target depth) is a logical explanation for why it takes longer for aDOT to stabilize.
The other statistical finding here that I hope opens up a more theoretical discussion is that, unlike the QB stability table, which showed remarkably consistent aDOTs across sample groups, aDOT for RBs clearly decreases with tenure.3 In other words, the longer a RB stays with a team, the lower his aDOT gets. The obvious question to ask here, which I don’t have an answer for at the moment, is “Is this a symptom of RB aging or a sample size effect?” In favor of the former is the sheer magnitude of the unmistakable aDOT decline in the table. Also in favor of the former is that the QB table didn’t show anything approaching such a decline despite smaller sample sizes in each “games played” group.
Returning from the theoretical back to the practical, below is a table showing actual and “true” aDOT for every running back in 2016 that played at least 25 percent of snaps per PFF:
|David A. Johnson||ARI||16||107||4.6||2||3.7||1|
|Jonathan C. Stewart||CAR||13||16||-0.1||53||0.3||53|
DT : IR :: TL : DR
Based on a split-half reliability analysis, I found that it takes 14 games (or about 30 targets) for a running back’s average depth of target to stabilize. I also found that there appears to be an aging effect, such that a running back’s depth of target decreases the longer and longer he stays with the same team.