Were you underwhelmed by the Eagles’ offensive performance against the Saints?  Readers here might not have been, but my guess is a lot of fans were.  This was one of the best offenses all season yet it scored just 3 TDs on 11 drives (and a FG).  You can probably guess where I’m going now:

Context.

What does a good offensive performance really look like?  Let’s look at a few stats to find out.  In the process, I think we’ll find a better perspective with which to judge all teams offensively.

How many points per drive should a good offense average?  Take a guess.  If an offense scores a TD every drive, it’ll average 6-7 points per, depending on the conversation choices.

How about 2.98?  Does that sound reasonable?  Barely less than a FG.  Or 1 TD every 2.3 drives.  Clearly, that’s a pretty good offensive performance.  However, broken down like this, it’s hardly spectacular.  Of course, as you probably suspect by now, that was Denver’s average points per drive this season; Denver also scored more points this year than any team in NFL history. (FootballOutsiders.com)

The MEDIAN this year was 1.91 points per drive.  That’s 1 touchdown every 3.66 drives (with no FGs).  Or, that’s 2 FGs ever 3.14 drives.  Imagine starting a game and having your first 3 drives go thusly: Punt, FG, FG.   Are you satisfied?  Probably not.  However, that’s 2 points per drive, which would place 12th overall this year.

Punt, FG, FG…repeat.   Congratulations, you’re a top 12 scoring offense.

The moral of the story is, offenses don’t score nearly as often as many people think they do.  It’s frustrating to watch a series of punts, and we inevitably start questioning the play-calling in such situations.  However, it’s a lot harder to score than people think.

The Eagles averaged 2.18 points per drive this season, good for 8th highest in the league.

Against New Orleans (the 10th ranked defense by DVOA), the Eagles scored 24 points on 11 drives.  That’s 2.18 points per drive…. In other words, the Eagles offense, on a scoring per drive basis, performed EXACTLY as expected.  In fact, it performed better than expected when you account for the strength of schedule difference between the regular season and the playoffs.

Meanwhile, New Orleans averaged 2.4 points per drive this season, 3rd best in the league.   Against the Eagles, the Saints also had 11 real drives (excluding kneels).  They scored 26 points.  That’s 2.36 points per drive, ALSO roughly in line with their season average.  Given the low ranking of the Eagles defense, it’s also reasonable to say the Saints should have been expected to score MORE.  That’s not really the point of this post, but I found it interesting nonetheless.

Forget about this “per drive” crap, what about TDs?

Good question.  How many TDs do you think the average team scores each game?

The Denver Broncos scored 4.4 offensive TDs per game this year (teamrankings.com).   Of course, that was the best offensive performance ever (a huge outlier).  In fact, that measure was roughly 40% better than the second place team…..the Eagles with 3.2 per game.

The MEDIAN team this year scored just 2.4 touchdowns per game.

The average # of offensive drives this year was 186.  That’s 11.62 drives per game.  If we use the median value of 2.4 TDs per game (to keep Denver from skewing), that means, roughly speaking, the average offense this year scored a TD on 20.6% of it’s drives.

1 touchdown every FIVE drives.  That’s average.  

Like I said…perspective….context.  Even the best offense ever, Denver this year, scored TDs on just over 1/3 of its drives.

Remember that the next time somebody complains about the offense’s “inconsistency”.

Time for another guest post from Jared.  For those of you who don’t know, Jared is my brother.  He’s also a U of Chicago MBA and a two-time Jeopardy! champion.   You can follow him @jaredscohen or see the original of this post at his site, linked here.  Now, his words:

Last year I put together some analysis of NFL kick returns. I was really motivated by one big question – Why do teams return kicks?

Initially, I wondered if returning kicks was even the optimal decision for teams trying to win football games. I wondered if the risks of turnovers and poor field position meant teams really should prefer a touchback to bringing the ball out of the end zone.

As a brief review, that’s not the case. Returning kicks is, on average, better for scoring points than taking a knee in the end zone as the returns leave your team with better field position. If you look at it in terms of expected points generated on kick returns vs. generated on touchbacks – the distinction is clear: (Note: this analysis relies on the concept of expected points based on field position – which I’ll assume readers have already seen and grasped)

This data comes from the first 16 weeks of this NFL season, over 2400 kicks. It’s also consistent with last year’s data.

So returning kicks is good, but think about why it’s a good idea. Although it presents better average field position, the average return nets only about four yards of position (and only two yards if the ball is brought out of the end zone relative to a risk-free touchback).

Linking back to material my brother has posted – the upside is directly tied to variance. Returning kicks is much more of a high-variance strategy.

Below – is an illustration of all returned kicks through Week 16 this year. The histogram shows the distribution of expected points.

You see the giant spike between 0.3-.04 which equates to a return between the 18 and 22 yard lines, that’s the most typical result (remember a touchback is worth 0.34 expected points). But there’s also an extremely long tail of positive performance, and these outliers can be worth a lot more (even a touchdown). Those outliers are what make kick returns worth the risks (injury, turnover), which is exactly what we mean when we talk about high-variance strategies.

A touchback has zero variance. That result is predictable and constant. But a return, that could be a whole bunch of possibilities.

OK – so let’s take the idea that returning kicks instead of taking touchbacks is a high-variance strategy as a hypothesis. Now, if that’s true, we would expect to see a couple different trends in the data. Generally, we would expect less talented teams to return kicks MORE often than their better opponents. Weaker teams should be pursuing higher variance plays in an attempt to pick up ground on those other (stronger) squads. In an example – you’d expect the Jaguars to try everything to beat the Broncos because Denver is extremely talented and playing a conventional game will leave the Jaguars at a big disadvantage. That could mean any number of things, more shots downfield, 4th down conversion attempts, surprise onside kicks, and we could expect – more kick returns.

So…is that something we actually observe in the data? Are weaker teams pursuing higher variance strategies in the form of more frequent kicks?

To test this, I went back and looked at my favorite kickoff metric – percentage of touchback eligible kicks returned. This counts the number of kicks that were returned out of the end zone as a proportion of the total number of kicks fielded in the end zone. Obviously – teams will return all kicks fielded short of the end zone, so we need to exclude these. The real decision point is whether or not teams bring balls out of the end zone – this is our true high variance strategic choice.

The data set it built off of play-by-play information, which is the best I can get. Unfortunately, there are a large number of touchback kicks where distance is not recorded and it isn’t specified whether the kick was fielded or kicked out of the end zone. After some initial eyeballing I’m confident these are kicks out the back of the end zone (Matt Prater of the Broncos had a lot of them as an example). So our set of kicks is a little smaller than you might expect. But there are still 950 kicks in our sample.

Then, I took all the NFL teams and split them into three performance tiers based on point differential. Teams with the highest point differential are members of the first tier, teams with the worst scoring differential are in the third tier. Below are the teams and their tier positions.

You can see the usual suspects in both the first and third tiers. And to me, this is where we’d expect to see the biggest change. These third tier teams – they have to do MORE to compete against first tier teams. Alternatively, first tier teams, one might argue, don’t need to take additional risk by sending their return man out of the end zone. If we look at touchback eligible kickoff return percentage across the different matchups – we can see if there’s any difference in the way teams behave. Do third tier teams return more kicks when they face off against first tier teams? Do first tier teams (who don’t need to pursue high-variance strategies) return fewer kicks?

Hmm…there’s almost no difference in return % whether the worst teams are facing other crappy teams or the best teams. That seems a little odd…as we had guessed the worse teams SHOULD be returning more kicks when they face better teams. This indicates that this doesn’t happen.

It’s also not a result of sample size, as most of these cells are large enough (80-120 observations).

As another check, I looked at touchback eligible return percentage relative to specific team talent (via point differential) on a team-by-team basis. I did this to see if there were any teams that really seemed to be demonstrating aggressive tactics at the individual level.

Again, this doesn’t appear to support our thinking that poor teams are pursuing higher variance strategies by returning more kicks. At best, it’s inconclusive. There are a couple of teams, like the Vikings, who really push the envelope – but there’s not a major correlation between team talent and return percentage (correlation is roughly -0.15)

Strange, but maybe identifying high-variance strategies before the game starts and following them blindly isn’t really what coaches of less-talented teams spend time on. Is there another way we can test our hypothesis?

Another theory is that if teams aren’t determining to return more kicks as part of pre-game strategy, maybe it’s something they pursue once they fall behind on the scoreboard. This wouldn’t even have to be exclusive to poor performing teams – any team that’s fallen behind might be more likely to run back kicks to try to break a big play to help catch up. What if we examine touchback eligible return percentage by in-game score differential?

The chart below illustrates the return percentage across a set of different score bands, ranging from down by more than 14 points to ahead by more than 14 points.

Again – there doesn’t seem to be any real connection between the scoreboard and aggressive kick return tactics. A team down by more than two touchdowns is just as likely to return a kick out of the end zone as one who is tied. If a kick return out of the end zone is indeed an aggressive play with a higher reward – teams don’t appear to be pursuing it MORE when they need to make up ground or LESS when they have a large lead. (As an aside, I absolutely cannot explain why having a small lead seems connected to a dramatic drop off in returns. I’ll chalk that up to some data wonkiness unless someone has a great insight there.)

But the broader concern remains. Shouldn’t teams which are behind or less talented need to take more chances to win? Why aren’t they doing that and bringing kicks out of the end zone?

My initial guess, though I’d welcome other speculation, is that teams the organizational structure of coaching almost inhibits something like that from happening. This comes with the obvious caveat that I’ve never coached in the NFL (so sure, Bill Belichick or someone else can dismiss all this out of hand as mom’s basement musings – but screw them). But if you’re the special teams coach of an NFL team – your work includes a thorough evaluation of your special teams and your upcoming opponent. All that work and planning becomes a little less valuable if a head coach just says – ‘Hey, I think we should return any kick we get in the end zone’

If the special teams coach is to maintain any kind of control over what his squad does – a simplistic rule like ‘run them out when we’re behind’ may not be sophisticated enough to justify all that pre-work and planning.

But that’s just a thought, based on the idea that coaches know their teams and customize approaches based on their own teams’ skills and the matchup with the opponent. Of course, when you actually look at the data, teams don’t really appear to be all that successful in managing their return game. Below is an illustration of touchback eligible return percentage, but this time charted against the average return position (i.e., return ability).

While we’d expect to see some correlation here – to show that teams with good return games return more kicks and teams with poor return teams take more touchbacks – that’s only true to the degree of a 0.2 correlation.

Some teams seem to get it – the Bills are really bad in the return game, but they rarely return kicks out of the end zone (on a relative basis – still over 50%). At the other extreme are the Vikings. The have Cordarrelle Patterson and, as such, they return kicks out of the end zone over 95% of the time!!!

On the flip side, look at Washington and St. Louis, teams with mediocre return units that run kicks out of the end zone 90% of the time. The Chiefs and Ravens seem odd as well – teams with great performance who could stand to run some more back. Now, maybe the Redskins are pursuing a high variance strategy, and maybe the Chiefs a more conservative one, but the overall results remain inconclusive.

At the end of the day, I come back to the idea of coaches and control over their special teams. For any team to read any of this and think about employing a ‘high-variance’ strategy – it really requires an admission of the role of chance in the outcome of a football game. Running every kick out of the end zone is a strategy based on the concept of inherent variability in outcome. Some returns may get stuffed, and others may go for big returns, but you can’t be sure when one or the other will happen. That view, to me, is fundamentally opposite the idea that with the right scheme and flawless execution – you can create the optimal outcome.

One of those ways of thinking supports the coach as the ultimate authority, while the other incorporates more probabilistic thinking. That gap is why I think we haven’t seen any patterns to support our hypothesis, and no clear evidence of high-variance kick return strategy consistently employed in today’s game.

I received some pushback from yesterday’s Billy Davis rant, so today I’m going to try to add some nuance to my explanation.  First, I want to note that while Davis bears the brunt of my criticism, he’s certainly not the only DC I disagree with on a fairly consistent basis.  Graded against everyone else, Davis is OK (for now).  However, as is usually the case, just because everyone else does something doesn’t mean we need to do the same thing.  Conventional wisdom, especially in sports, frequently lags the “optimal” strategy.

So….Defensive Strategy, and more specifically, the Blitz.

My general take on this is that the Blitz (sending more than 4 pass rushers) should be viewed as a TOOL, not a general philosophy.  I realize that in Philadelphia, that’s borderline heresy (lot of Jim Johnson fans out there).  But let me explain.

Two Factors

To complete a pass, two things must happen (generally speaking): an “open” receiver must exist, and the QB has to identify that opening (after which he presumably throws the ball there).  It’s tough to determine what constitutes an “open receiver”, so I’m going to discuss this side of things in terms of Windows.  So a passing window refers to an opportunity to place the ball where the receiver can catch, and one must exist and be identified in order to complete a pass.  Simple enough?

Also, for a QB to identify the available passing window, he must have TIME to do so.  The more time he has, the higher the odds of him seeing an existing window or of one developing.

By breaking the process down into these factors, we can see the basic trade-off in defensive strategies (against the pass).  The best of both worlds, of course, is to minimize the passing windows AND minimize the time the QB has to identify them.  That’s why DEs are so coveted.  If you can generate a strong rush (i.e. lower QB time) with just 4 d-linemen, you can use everyone else to close passing windows.  However, very few teams area able to do that on a regular basis.

More often, you have to make a choice.  You can rush an extra man (blitz), which should decrease the amount of time the QB has to see a window.  Conversely, you can rush fewer men, and use more of them to minimize the windows.

With me so far?  Good, now let’s talk a little bit about passing windows.

Passing Windows

Passing windows open and close throughout each play.  A complete pass occurs when one of them opens and the QB hits it.  To help illustrate, I’ll pick a random frame from Sunday’s game:

Nice….Now let’s superimpose the passing window on it.  Despite the outcome of the play (Boykin game-saving interception), there was, in fact, a window to hit here for Orton.

That’s a rough approximation, obviously, but you can see the idea.  Given such a big opening, how the hell did Orton miss?  Any guesses?

How about:  He’s not a good QB!?

That’s a little unfair (just a little), because every QB misses opportunities sometimes.  However, let’s dig a little deeper into this.

When deciding what pass-rush strategy to use, there’s perhaps no greater factor for consideration than the skill of the opposing QB.   We now have to combine the QB skill with our Window illustration from above.  Let’s visualize it like this:

Don’t get too caught up in the relative sizes, this is far from a “to-scale” illustration.  In the middle we have the passing window.  On either side I’ve provided a visual representation of each QB’s (Peyton and Orton) accuracy.  Think of the two layers as confidence intervals; something like 70% certainty the ball will end up within the smaller red circle and 90% certainty it will end up within the outer circle’s boundaries.

Hopefully this is rather intuitive.  Now play a mental game using those images.  The green square will move across the screen from left to right.  You control the red circle, and your job is to align it with the passing window and and press go.  Think of it like aiming a rifle.

Now…which player’s range (red circles) would you rather play with?

Easy, Peyton Manning’s, because his confidence ranges are smaller, meaning there’s a smaller margin of error.  For example, let’s say you align each perfectly with the passing window:

See the problem?  The window is smaller than Kyle Orton’s accuracy range.  Meanwhile, it the window is significantly larger than Manning’s 70% accuracy range.  The upshot, naturally, is that Manning is a lot more likely to complete this pass.  Going back to our game image above, we can visualize the pass like this, with the yellow X denoting the final placement of the ball, which was intercepted.

Moreover, we can extrapolate the idea.  In general, Peyton Manning will be able to hit smaller windows than Kyle Orton will.  Obviously, smaller windows occur more frequently than larger windows, hence Peyton Manning, by virtue of his accuracy, will have many more opportunities to complete passes than Kyle Orton.

Now pretend you’re a defensive coordinator.  Remember you have a choice to make between eliminating passing windows and minimizing time.  In this exercise, you cannot do both.  Against which player is the “window elimination” strategy more likely to work?

Easy (again), Kyle Orton.

Since Peyton Manning’s required window size is so small, eliminating them will require extremely good coverage.  More likely, you can play excellent coverage on the receivers and STILL not prevent several of these small windows from opening up.

Conversely, against Kyle Orton, things aren’t so difficult.  He needs a relatively large window.  Large windows are easier to eliminate.  You don’t have to play perfect coverage. Notice in the Boykin play, there was a relatively large are in which Orton could have complete a pass for a big gain.  Fortunately for the Eagles, Orton didn’t hit his spot.  Boykin’s coverage was far from perfect, but it didn’t need to be!

In light of that, go back you your strategy decision.  Which do you think is easier to do:

– Eliminate windows

– Minimize time

Now consider that Orton was operating out of 3-step drops for much of the night.  Then, the answer is easy.  Eliminating his passing windows is the much higher-probability play.  Note that’s the BASE strategy.  Obviously, you need to blitz every once in a while, if for no other reason than to add some unpredictability.

That’s the crux of my argument against Billy Davis’ blitzes.  He doesn’t seem to vary his usage as much as I believe he should, and he doesn’t save his blitzes for high-leverage situations.  Instead, he uses them in LOW-leverage situations, where the reward of a sack is comparatively low, especially when weighed against the odds of a big play.

Against a great QB (like Drew Brees this week), you have to be much more aggressive because it’s much more difficult to eliminate those passing windows.  Moreover, there’s another factor to discuss:

The Blitz Bonus

A very successful blitz will result in a sack.  A sack dramatically swings the odds of a turnover (punts included) in the defense’s favor.  Now, comparing opposing QBs, against which ones do you think that’s most important to do?  I’ll give you a hint, it’s not Kyle Orton.

Against a great offense or great QB (frequently one and the same), the odds of allowing a 1st down are comparatively high.  For example, according to Pro-football-reference.com, the Denver Broncos faced 93 third downs needing 5 or fewer yards for a first down.  They converted 62.4% of those.

Now compare that to a bad offense, like Baltimore (ranked 30th by Football Outsiders).  The Ravens faced 96 third downs with 5 or fewer yards to gain.  They converted just 49% of those.

As you can see, getting to 3rd and less than 5, normally not considered much of a “win” for the defense, is still good enough to get you to 50/50 against a bad offense.  Assuming each opportunity is an independent event, the odds of the Ravens converting two consecutive such third downs is just 25%.

Hopefully your mental light-bulb just turned on.  Facing a Kyle Orton-driven offense, the Eagles were looking at a team much closer to the Ravens than the Broncos.  In that situation, just preventing a big play and forcing the Cowboys to convert a string of third downs was VERY LIKELY to produce a punt.

In other words, we didn’t NEED a sack!  The odds were already in our favor.  Conversely, if we had been facing the Broncos, the risk/reward equation flips.  That team is much more likely to convert a string of third downs, meaning the defense needs to do something to increase its odds.  Getting a sack is one of the only affirmative ways to do this.  In that case, the reward of getting a sack outweighs the risk of giving up a big play.  Without the sack, you’re likely to give up a long drive anyway!

Against a bad offense, though, that’s not the case.  It’s better to sit back, eliminate passing windows, and wait for the odds to shake themselves out.  By blitzing bad QBs, you’re making a foolish grab for upside that you don’t need.  Bad QBs will struggle to hit receivers that are even marginally covered, so why make it easy for them by making those passing windows larger?

Wrapping Up

Hopefully that illuminated things a bit more clearly.   Basically, against bad QB’s, the odds are already in your favor.  The reward of a sack (or forced incompletion), and the increased odds of a punt that come with it, are NOT worth the risk of the big play. In all likelihood, a bad offense will NEED a big play in order to score.  They simply won’t be able to string together a 12 play drive with a lot of 3rd down conversions.  Hence, the goal should be to get to third down as often as possible, and let the odds take effect.

Against a great QB, though, that’s not enough.  They ARE somewhat likely to string together 3rd down conversions, especially if their short yardage situations.  Similarly, they DO NOT need a big play to sustain a drive.  In that case, the risk of giving up the big play (which is worth relatively less to a great offense than to a bad one) is worth the associated reward of a longer yardage situation (which the defense needs to push the odds in its favor).

That doesn’t mean you never blitz a bad QB or always blitz a great one.  It does mean that you’re general pass rush strategy, particularly when it comes to sending extra pass rushers, should vary greatly depending on which QB you’re playing against.  Just saying “we’re a blitzing defense”, so we’ll blitz, is a very low-level strategy.  It’s far too simplistic, and sounds a lot more like a crutch than a well-thought out, adaptable and deployable strategy.