When Does Yards Per Attempt Stabilize?

Through Week 1, Matthew Stafford leads all quarterbacks (QBs) with 10.81 Yards per Attempt (Y/A) and the league average is 7.12 Y/A. Of course, 32 attempts does not a season make, so anyone who’s ever heard of regression to the mean realizes Stafford probably won’t be able to sustain his current pace. Or, in the language of measurement theory, it’s highly unlikely that his current Y/A performance is a reliable indicator of his “true” Y/A ability. At some point, however, it will be, so today’s mission is to answer the question “When does Y/A stabilize?”

Now, I’ve ranted before about how we shouldn’t be reinventing the wheel in NFL analytics, and Monte McNair’s already established the stabilization point of Y/A to be approximately 400 attempts. But before you shame me for my blatant hypocrisy, let me just say that (a) I’ve improved his analysis in several ways; and (b) this is part of a much larger project that, in time, you’ll see far exceeds the scope of McNair’s work.

Methods

It’s been a while, so here’s a reminder of what the analysis entailed:

1. I collected data for all QBs with at least 400 total pass attempts from 2002 to 2013.
2. To control for team effects, I included only those QBs that had 400+ (or 500+ or 600+, etc.) attempts for the same team.
3. Starting with QBs that had 400+ attempts, I randomly selected two sets of 200 attempts for each QB, and calculated the correlation (r) between the two sets.
4. I performed 25 iterations of Step 2 so that r converged.
5. I repeated Steps 3 and 4 for QBs with 500+, 600+, 700+ attempts, and so on.
6. For each attempts group, I calculated the number of attempts at which the explained variance, R², would mathematically equal 0.5. ((The formula is (attempts/2)*[(1-r)/r].))
7. I calculated a weighted average of my Step 6 results. ((Weighted by group size.))
8. I calculated the “true” Y/A for a hypothetical QB that’s posted a 7.00 Y/A through X number of attempts.

Results

Here they are:

Attempts	n	r	R2 = 0.50	Avg Y/A	Obs 7.0 Y/A
200	123	0.31	455	7.12	7.08
250	106	0.38	413	7.14	7.09
300	89	0.44	385	7.15	7.08
350	81	0.49	364	7.16	7.08
400	74	0.52	366	7.19	7.09
450	66	0.53	398	7.19	7.09
500	60	0.57	372	7.22	7.09
550	52	0.58	396	7.26	7.11
600	47	0.63	355	7.28	7.10
Wtd Average			396	7.18	7.09

As a reminder, here’s an example of how to read the table. There were 52 QBs who had (at least) two sets of 550 attempts for the same team, and those QBs had an average Y/A of 7.22. Given their split-half correlation of 0.58, Y/A stabilized at 396 attempts for this group. And given their 7.26 average Y/A, we can estimate that a QB with 7.00 Y/A after 550 attempts has a “true” Y/A of 7.11.

The “Wtd Average” row is the bottom line of the analysis, both literally and figuratively. Mimicking McNair’s previous results, I found that Y/A stabilizes at 396 attempts (aka “approximately 400”). Furthermore, we can use this total along with the league-average Y/A to estimate that a QB with 7.00 Y/A after 396 attempts has a “true” Y/A of 7.09.

And just to call attention to something I’ve glossed over in the past, take note that 7.09 is halfway between the hypothetical QB’s 7.00 Y/A and the average QB’s 7.18 Y/A. This is due to the underlying math that allows for our “half-skill/half-luck” conclusion about Y/A stability. ((The formula is [(7.00 * 396) + (7.18 * 396)]/ (396+396) = 5,615.28/792 = 7.09.))

Application

I’m not going to be posting what follows every week, but it’s an example of what you can do with reliability analysis in terms of practical application.

Now that we know Y/A stabilizes at 396 attempts, and now that we have passing data from Week 1, we can start looking at real QBs rather than hypothetical ones. All we have to do is add 396 attempts of average Y/A performance to what each QB produced in Week 1. Doing so results in the following “True” Y/A rankings: ((The table is fully searchable and sortable, and you can change the number of displayed entries.))

QB	Tm	Obs Y/A	Rk	True Y/A	Rk
Matt Ryan	ATL	10.42	3	7.44	1
Ben Roethlisberger	PIT	10.74	2	7.41	2
Matthew Stafford	DET	10.81	1	7.40	3
Ryan Fitzpatrick	HOU	9.36	4	7.24	4
Carson Palmer	ARI	8.22	7	7.21	5
Colin Kaepernick	SF	8.74	5	7.21	6
Drew Brees	NO	7.93	9	7.20	7
Jake Locker	TEN	8.06	8	7.19	8
Andy Dalton	CIN	7.92	10	7.19	9
Austin Davis	STL	8.35	6	7.19	10
Geno Smith	NYJ	7.89	11	7.17	11
Tony Romo	DAL	7.59	13	7.16	12
E.J. Manuel	BUF	7.86	12	7.16	13
Peyton Manning	DEN	7.47	14	7.15	14
Brian Hoyer	CLE	7.42	15	7.14	15
Robert Griffin	WAS	7.22	16	7.13	16
Nick Foles	PHI	7.16	17	7.12	17
Jay Cutler	CHI	7.12	18	7.12	18
Andrew Luck	IND	6.98	19	7.10	19
Michael Vick	NYJ	0.00	34	7.10	20
Matt Cassel	MIN	6.80	21	7.10	21
Russell Wilson	SEA	6.82	20	7.10	22
Derek Anderson	CAR	6.76	22	7.09	23
Shaun Hill	STL	6.23	24	7.09	24
Philip Rivers	SD	6.61	23	7.08	25
Chad Henne	JAX	6.19	25	7.03	26
Aaron Rodgers	GB	5.73	27	7.01	27
Alex Smith	KC	5.77	26	7.01	28
Ryan Tannehill	MIA	5.56	29	7.00	29
Josh McCown	TB	5.23	30	6.97	30
Eli Manning	NYG	4.94	31	6.95	31
Derek Carr	OAK	4.72	32	6.94	32
Joe Flacco	BAL	5.56	28	6.91	33
Tom Brady	NE	4.45	33	6.79	34

The first thing that jumps out is that the Top 3 in observed Y/A (“Obs Y/A”) are flipped in the True Y/A rankings. That’s because, although they produced similar Y/A stats, Ryan’s came over more attempts (43) than did Roethlisberger’s (34) and Stafford’s (32). Mathematically speaking, Ryan’s calculation was based on a regression to the mean of about 90%, ((90% represents 396 of the 396 + 43 = 439 attempts in the denominator.)) whereas Roethlisberger’s and Stafford’s were based on about 93% regression to the mean.

A couple of other QBs who stand out are Michael Vick and Joe Flacco. Vick’s 0 pass yards against Oakland came on only 1 attempt, so practically all of his 0.00 observed Y/A is regressed towards the Y/A of an average QB. On the flip side, Flacco’s 5.56 observed Y/A was the 28th-ranked Y/A performance of Week 1 and it featured the most attempts (62). That combination resulted in an even lower True Y/A ranking despite gaining over a yard’s worth of it thanks to regression — or in this case progression — to the mean.

DT : IR :: TL : DR

Matthew Stafford had a great game against the Giants. But while he was able to sustain 10.81 Y/A over 32 attempts, he’s unlikely to as the season progresses. How many more attempts do we have to see before his Y/A becomes a reliable indicator over the long term? Well, in virtual lock-step agreement with previous research, I found that it takes 396 attempts before a QB’s Y/A represents 50% of his “true” Y/A ability and 50% luck.