The Defensive Line and the Secondary, Part II

When I first delved into the connections between pass rush and pass defense, I came away conclusions fueled partly by data, and partly by ignorance.  They were:

We have found that there is no inverse correlation between that pass rush effectiveness and passing effectiveness; on plays where a sack does not occur, per-play pass yardage is unaffected.

We have found that there is no inverse correlation between interception rate and passing effectiveness; on plays where an interception does not occur, per-play pass yardage is unaffected.

These may have been exaggerations.  Commenters, emailers, and strangers on the street have since filled me in.  The correlation coefficients looked dismissible to my amateur eyes, but weren’t—not entirely, anyway.  There was a –.068 correlation coefficient between “dropbacks per sack” and “raw yards per attempt”.  Essentially zero, I thought, but it’s closer to –.1 than to zero—and that’s more than nothing.  Not much more, but more.

The correlation coefficient between "attempts per interception" and "yards per attempt" was –.091.  That’s still only approaching .1; a weak correlation at best.  Still, my two statements above aren’t entirely accurate.  Over the past 20 years of NFL data, there has been a very mild inverse correlation between sacks, and pass effectiveness on non-sack downs (and the same goes for INTs).

This all implies that the pass rush might indeed affect offense even when it doesn’t hit home—but I couldn’t prove it with the data I had.  I’d been working with the 50,000-foot-view: year-over-year leaguewide NFL trends; he low resolution of this data may have blurred the very effects I’m hoping to measure.

As I said in my “parallel efforts” post last week, the Mathlete, of MGoBlog fame, did some of that further research.  As the Mathlete compared sacks, picks, and overall performance, he found:

Not entirely surprisingly, the better a defense is at producing sacks and interceptions, the better it is on downs where neither occur. For every point per game that a defense generates due to sacks, the overall pass rush generates 1.2 ppg of additional value. Interceptions are also powerful, but not as much so. Each ppg of value a defense generates through interceptions is worth 0.9 ppg of additional value.

This relies on “expected points,” here explained by Brian Burke of Advanced NFL Stats.  EP is a way to recognize the reality that four yards gained on 3rd-and-3 are not nearly as valuable as four yards gained on 3rd-and-8.  By finding the historical likelihood of scoring on a given down and distance from a given field position, Burke has created baseline EP values.  By finding the difference between the EP values from before a play to that play, you can gauge what that play is worth (in EP Added, or EPA).

Using EP allowed the Mathlete to quantify exactly what sacks—and the associated pass rush on non-sack downs—are worth.

From 2006-2009 in games between two D1 teams in competitive game situations (the “universe” for this and most of my analysis) the average defensive unit produced 2.3 ppg worth of sacks and 2.0 ppg worth of interceptions.

So, an average defense produced 2.3 ppg worth of sacks, and gained 2.76 ppg worth of “footsteps effect” from the pass rush.  Likewise,  the average defense snared 2.0 ppg worth of interceptions, and 1.8 ppg worth of “dink-and-dunk effect”.  Mathlete concluded that all he found was the dog wagging the tail: sacks and picks are merely indicators of good defensive performance, not the other way around.

However, this is all still limited to sacks, and attempting to derive the “cringe effect” on non-sack plays.  However, I’ve pulled down data from ProFootballFocus.com. They use film review to grade individual performances, as well as capture unofficial or untracked stats—like QB hits, and QB pressures.  Rather than attempt to measure the size of the tyrannosaur by the size of the ripples in the water cup, I’ll look at the actual data.

The data I’m using is PFF’s player data for the 2009 season, aggregated into team numbers.  A caveat, these numbers do vary from the official NFL totals, especially with subjective stats like sacks, half-sacks, and tackles.  From what I’ve seen, the differences are small, but it’s worth getting out there.  Anyway, the first order of business: test my assumption that pressures correlate with sacks.

Result?  “Yes,” but not as strongly as I’d assumed.  The correlation coefficient was .584, quite strong.  But as an anonymous commenter noted:

One of the best indicators of nature of the relationship is the effect size calculation. You can obtain an effect size by squaring your correlation coefficient. This will tell you the amount of variability the variables contribute to each other. There's a lot of noise in NFL data so an effect size will tell you how much unique data is explained by the correlation alone.

Rule of thumb for effect sizes: small .2; medium .5; large .8

The effect size here is displayed in that graph as R-squared: 0.3413.  It’s far from one-to-one, but it’s safe to say that teams that get lots of pressures tend to get lots of sacks, and vice-versa.  For what it’s worth, the best “finishers” are the Redskins, getting 43 sacks to 109 pressures.  For academics’ sake, the Falcons were the other way around: 160 pressures to only 31 sacks.  The Lions were . . . well, terrible: 23 sacks, and 101 pressures.

Next up, let's take that “hit” and “pressure” data out for a spin:

This is the total yards per attempt allowed, regressed against the percentage of dropbacks where “pass rush” was generated.  Dropbacks are “attempts + sacks”, and I’m dividing that number by cumulative sacks, hits, and pressures.  The resulting correlation is weak, –0.152, and when squared to get the effect size, it’s a negligible .0232. 

So, once again, we’re seeing no real connection between pressure and good pass defense.  You see the most interesting data point?  Yeah, that one—the one where they’re allowing three-quarters of a yard less per pass attempt than the 2nd-best team.  That’s the New York Jets; they allowed a miniscule 4.91 yards per pass attempt, while finishing 19th in PFF-scored sacks, and 9th in per-dropback “pass rush” rate.  So, the Jets had a good pass rush, but nothing that correlates to the incredible per-play passing effectiveness seen here.

Secondly, my eyes were drawn to the third-rightmost data point, and how we’d have to scrape it off the Yards-per-attempt ceiling.  That would be the Cleveland Browns, whose 47 sacks, 42 QB hits, and 139 pressures generated pressure on 42.93% of dropbacks.  That pass rush rate is third-best in the NFL, behind only Dallas (46.91%) and Minnesota (44.77%).  Yet, Cleveland’s pass D was flat terrible: it allowed 7.44 yards per attempt, ahead of only Oakland, St. Louis, Miami, and, yes Detroit (7.80). 

Things are not looking great for our thesis.  If the Lions improved their 2009 per-dropback pass rush rate to match Cleveland’s, from 27.85% to 42.93%, we’d certainly expect a better improvement in allowed Y/A than 7.80 to 7.44.

Last time, passes defensed were to coverage what sacks were to pass rush; I presumed that in years where passes defensed were up, “coverage” was better.  In this case, though, I have Pro Football Focus’s “thrown at” and “reception allowed” figures.  Again, let’s dispense with trying to derive the whole picture from one rare subset of outcomes.  FYI: Total TAs don’t equal attempts, and total receptions allowed don’t equal total receptions.  I presume the discrepancies are from uncatchable passes, screen and swing passes that aren’t thrown “into coverage”.

I’ll divide total per-team receptions allowed by total per-team TAs to get a per-team Percent Caught, as PFF does on a per-player basis.  This should be an actual measurement of the quality of pass coverage.  Let’s regress it against Y/A:

Bingo.  There’s as strong of a correlation between percent of catches allowed by coverage and yards-per-attempt, as there is between pressures and sacks.  Oh, and who’s that team with the astonishingly low percent-caught, and correlatedly low Y/A?  Yup, it’s the Jets.  The evidence is mounting: pass rush doesn’t equal pass defense, pass defense equals pass defense.

But, that’s not the whole story, is it? After all, isn’t the magic of a 4-3 defensive line supposed to be that it can generate pressure without a compromising coverage?  Won’t there be great improvement in a 4-3 that generates pass rush from the line, without having to blitz linebackers and DBs?  Stay tuned for the next exciting adventure!

Technorati Tags: nfl,detroit lions,defense,sacks,interceptions,statistics,analysis