The Defensive Line and The Secondary

Lions fans worrying about the secondary have been closing their eyes, clicking their heels, and chanting, “The defensive line, the defensive line, the defensive line.”  They’re telling themselves the additions of Kyle Vanden Bosch, Corey Williams, and Ndamukong Suh will drastically improve the pass rush, thereby shortening the field for the secondary.  They’re telling themselves that opposing quarterbacks won’t have time to attack Chris Houston, Jonathan Wade, and Amari Spievey. 

Are they telling themselves the truth?  Can an improved pass rush really hide substandard secondary play?  How much better would the defensive line need to be to cover up the lackluster coverage?

First, I went to my favorite NFL stats database, Pro-Football-Reference.com.  I decided to go back to 1982, the year the league started keeping track of sacks as an official stat.  I started with my favorite metric of passing efficiency, net per-play yards per attempt.  The idea, if I understand correctly, is to measure the mean yards gained on every dropback.

So, my first task was simple: plot the correlation between sacks, and net-per-play yards attempt.  This should show us if there is indeed a correlation between pass rush effectiveness (expressed in sacks), and depressed passing efficiency.  There is a critical assumption, though:

• We assume that if sacks per dropback are significantly up or down across the NFL, there has been an increase or decrease in total pass rush effectiveness that season.

Wow, that looks pretty good right there!  It sure looks like there’s a nice little cluster around the trend line, and . . . wait.  What’s that one lonely data point all down there by itself?  Ah, it’s 1982, the strike year.  That explains it: since they only played nine games, most stats were just about halved, including sacks.

. . . wait.  I used the wrong values.  The total, leaguewide number of sacks will vary with number of attempts.  In years where teams ran fewer plays, or passed less often, the number of total sacks would be lower—but that wouldn’t mean there was a decrease in pass rush effectiveness.  No, what we want is sack rate; how often running a pass play results in a sack.  I switched the X axis to average number of attempts per sack, and tried again.  Remember, the hypothesis is that in years where sacks occurred more often, NY/A should be depressed.

The coefficient of correlation here is .451.  For those of you who, like me, remember nothing of the high school math you didn’t understand at the time anyway: zero is no correlation at all, and 1 is perfect correlation.  I’m not a statistician; I don’t know how to determine if this correlation is “statistically significant,” quote-unquote, but it seems fairly strong, and is definitely nonzero.

However, we have a problem: “Net Yards per Attempt” includes sacks.  It’s attempts plus sacks, over passing yards minus sack yards.  The higher the sack rate, the more passing yardage will be depressed—so our correlation is artificially enhanced.

Removing sack yardage from the equation, here’s the same chart:

Now our chart looks nearly random.  The correlation coefficient is .068; approaching zero.  Our hypothesis, that there is a correlation between increased pass rush and depressed passing effectiveness, is in rough shape.

There’s some data I’d really like to have that I don’t, namely “QB hurries” or “QB pressures”. These are plays where the pass rush was disruptive, but didn’t result in a sack.  Unfortunately, that data isn’t official, and is likely to be quite subjective even if we had it.  So, going forward:

• We have found that there is no inverse correlation between that pass rush effectiveness and passing effectiveness; on plays where a sack does not occur, per-play pass yardage is unaffected.

However, we know that there is such a thing as "good pass defense."  If a pass rush that’s getting more sacks isn’t it, then what is?  The first answer that springs to mind is “a defense that gets lots of interceptions,” right?  So, here’s interception rate per attempt, against raw yards per attempt:

Ugh, another scattershot plot.  The correlation is –.091, again nonzero but not strong.  Again we see: interceptions are certainly a good thing to get, but a defense that gets a lot of interceptions isn’t necessarily a “good pass defense”.  Just as with sacks, when you’re not actually picking a pass off, a high interception rate doesn’t help you much.

• We have found that there is no inverse correlation between interception rate and passing effectiveness; on plays where an interception does not occur, per-play pass yardage is unaffected.

So again, if there’s such a thing as “good pass defense,” is there a stat that directly correlates to it?  Yes, I think there is: passes defensed.

Unfortunately, it's only been tracked since 2001—and in 2002, the total number of passes defensed, league wide, was about half of what it’s been in every other year.  I don’t know  what happened there, but you can see it on the chart:

Yeah, that’s an outlier.  Given that it was only the second year of tracking the statistic, it’s not unexpected.  We drop it, and:

Ah-HA!  The correlation coefficient is .495, and we see a clear trend emerging.  We only have eight data points, but there’s correlation that isn’t built-in: on plays where the quarterback threw the ball forward, the more frequently passes were broken up, the less effective pass offense was, and vice versa.

One might point out that a pass defensed is an incomplete pass, so greater numbers of them necessarily lower YPA—but  the same is true of interceptions, and we saw no such correlation above.  The sample size is also troubling, in terms of establishing a statistical trend—but remember, each data point represents every passing attempt in the NFL for an entire season.

• We have found that there is a correlation between passes defensed and passing effectiveness; on plays where a pass is not defensed, per-play pass yardage is still depressed.
• We can assume that if passes defensed per pass attempt are significantly up or down across the NFL, there has been an increase or decrease in total pass coverage effectiveness that season.

We're left with the depressing conclusion that the only good pass defense is good pass defense.  However, that's not really the case, either.  Sacks and interceptions, though they don’t affect the interplay of pass offense and pass defense outside of themselves, are still extremely important in terms of total defense.  Stopping drives and preventing scoring is the primary job of a defense; a third-down sack or a red-zone INT can erase sixty or seventy yards’ worth of Montanaesque passing effectiveness.

What's happening here?  Follow the blue line, passing YPA.  See the big dip it took in 2003?  And the subsequential huge spike in 2004?  That was a result of the Ty Law Rule, a change in the enforcement of Pass Interference and Illegal Contact rules.  It’s named for the extremely effective, physical press coverage that Ty Law and the Patriots popularized in 2001, 2002, and 2003.  The refs were “letting them play,” and the result was that passing was depressed.

In 2004, though, the refs started throwing the hankies, and passing effectiveness exploded as the press corners backed off.  As teams could no longer afford to let their corners maul wideouts one-on-one, more safeties had to be rolled over to help.  As more safeties rolled over to help, tight ends, backs, and slot receivers flourished.  As more tight ends, backs, and slot receivers have flourished, more and more teams have switched from 4-3 fronts and man-to-man coverage, to 3-4/4-3 hybrids and zone coverage.

The upshot is that passing is getting more and more effective. Schemes are more complex, quarterbacks are sharper, and multi-back, multi-WR sets—difficult to defend with a traditional press defense—are forcing defenses to attempt aggressive, flexible disruption, with a broad safety net, rather than lining up 11-on-11 and winning battles.

What does this mean for the Lions?  Likely the obvious: we’ll see lots more pressure, including lots more sacks, and that will improve the scoring defense—though not by nearly as much as is needed for the Lions to be a consistent winner.  Ndamukong Suh and Kyle Vanden Bosch will not make the secondary any less porous, and teams—when they get a pass off—will still torch the Lions for long gains with great regularity.

Real statisticians, please, please, enlighten me in the comments.

UPDATE: I’ve followed this article up with Part II and Part III, drilling down much farther and finding really interesting stuff; check them out.

Technorati Tags: nfl,detroit lions,defense,safeties,sacks,interceptions,statistics,analysis