The Watchtower Review

The point of the Watchtower posts was to forecast the performance of the Detroit Lions against their weekly opponents. From the start, I’ve used historical performance data of the Lions coordinators against their opposition’s. By controlling for the relative talent of the players, I tried to isolate systemic advantages at the X-and-O level. I then tried to apply those advantages to the teams’ current skill levels, and project a result.

The Watchtower is one of my most popular features; people really dig it. It’s fun to write, especially researching every coordinator’s coaching tree, and picking the picture. However, after three years, I’m no longer satisfied with The Watchtower an alternative “game preview,” or as a predictive tool.

Watchtower Problem #1: heavy reliance on per-game team averages.

When I use average yards per attempt and average yards per carry, it gives a pretty accurate picture of those players’ performance levels. Whether a quarterback has 25 or 50 attempts, or 200 or 400 yards, dividing one by the other tells you at what rate the quarterback is generating offense, every time. But dividing “points scored in a season” by “games in a season” doesn’t work. A “game” is not a fixed unit of measure; there’s a wide variance in the number of possessions and plays in a “game.”

In every pass attempt, there is exactly one pass attempt, one bite at the apple. In every game, there’s a wide variance in possessions, time of possession, and plays. Example: when the Lions hosted the Vikings, they scored 34 points. When they hosted the Chargers, they scored 38. On the face of it (and in terms of the “points per game” numbers I’ve been using), the offense was very effective in both games.

However, in that Minnesota game the offense netted just 280 yards and 20 points from ten possessions. Against San Diego, the offense netted 440 yards and 31 points from eight possessions. This is a massive difference in effectiveness and it’s almost completely uncaptured by the current Watchtower methodology.

Dropping the "per game" team averages would allow me "tell the story" more effectively; I thought there was a very high chance that the first Packers game would be shockingly conservative—and the rematch a track meet. But using season average against season average, there’s no way to project either of those outcomes.

Finally, that "track meet" effect means something: there is a tendency for points to follow points, and that speaks to a very real offense/defense interaction effect that isn’t accounted for, either in traditional analysis or in The Watchtower. When one offense puts the pedal to the metal, the other one follows—and both defenses, apparently, just let it happen. Why? What’s going on here?

Watchtower Problem #2: No real accounting for turnovers or special teams.

This is one that’s bothered several readers from the get-go. The Watchtower is a study of offense-defense interaction: what happens when offensive scheme A meets defensive scheme B. But special teams and turnovers play a huge role in the final score.

In the Thanksgiving Day game, when the Lions and Packers played to a stalemate for most of the first half, a tipped pass fell into enemy hands and the Packers’ offense got to start deep in the heart of Lions territory. That was the game-changing play both teams desperately needed. Despite incredible down-to-down play by the defense, the offense was really the unit that put the Packers in position to score.

On special teams, the Lions’ coverage units struggled mightily throughout the first two thirds of the season, and it regularly hung the defense out to dry. Moreover, the iffy upfield blocking for Stefan Logan (and his own iffy fair catch decisions on kickoffs) failed to make the field shorter for the offense.

Watchtower Problem #3: The Human Element.

I project ranges for points, passing effectiveness, and running effectiveness for each side—then basically use the “Mitigating/Aggravating Factors” and “Conclusions” section to winnow those down to the final score I deem “most likely,” usually via talking-out-loud thought experiment.

There are several layers of my own bias involved here—and even though I work hard to follow where the data leads me, a little bias on top of a little bias on top of a little bias makes a big difference. I can definitely lead the statistical horse to water if I want to—and sometimes I do even when I’m trying not to.

What I’d love to be able to do is project a range of possible outcomes and their probabilities, so when I say “The most likely outcome is . . .” my a hand won’t be moving the data’s mouth.

Thoughts?