Does interception rate persist? Potential red flags for next season

The Eagles had a very successful 2013 season.  Now we need to evaluate it.  After a success, particularly one as resounding as we experienced this year, the most important question to ask is:

Were they lucky or good?

Obviously, if they were very lucky, then success next year is less likely.  Heading into this season, I was one of the few Eagles writers/bloggers to predict anything resembling what actually happened (I had them at 9-7, but I was just 20 points off on the point differential).  Part of the reason I was so bullish was that the Eagles had bad luck last year, especially as it relates to turnovers.

Now, we have to take the same view of things.  Today, I’m going to focus on one particularly important statistic from this season:

Nick Foles has an interception rate of 0.6% this season.  (The single season record is 0.4%)

Aside from the obvious (interceptions are bad), this carries additional weight because if factors into whether or not Foles can be a “franchise” QB.  I personally do not think he’s a great QB (or likely to become one).  However, I do think he’s “good enough”.  The other side of the argument is that he lacks any truly elite skills.  Most apparently, his arm strength isn’t great and he’s slow.  His accuracy seems very good, but it’s much harder to judge that type of attribute than something more measurable like strength.  As a result, while watching him play, it’s much easier to focus on what he CAN’T do or isn’t doing than on what he is doing.  In light of that, allow me to posit the following:

It’s possible that Nick Foles’ “elite” quality is the ability to avoid interceptions without abandoning downfield throws.  It’s possible he just has an excellent internal sense for the risk/reward of each throw.  Or, if you think back to my blitz post (windows v. time), he might just have a very good sense of when a window is large enough for his skill level.

If that’s true, than I don’t see any reason why he can’t be an “elite” QB.  Of course, we don’t know if that’s true and, on balance, it seems unlikely.

Today, let’s take a very preliminary step towards testing it.  As the title suggests, I think the best way to proceed is to see if Interception Rate persists over time.  In other words, how much does a QBs interception rate one season tell us about his rate the following season. If it does persist, then avoiding interceptions is likely a skill and we can feel really good about Nick Foles.  If it does NOT, then we’re in trouble, because Foles’ amazing statistics this year were built primarily upon not throwing interceptions.

The Sample

There are a number of issues with trying to test interception rate persistence, so before we even get close to a result, we need to remember everything here is just informative rather than solid proof (I’ll explain the problems below).

To get a preliminary look, I selected 13 active QBs.  The only prerequisite was that they had to have started for at least a few years.  Of course, this introduces our first source of bias, survivorship.  However, we’re looking at persistence, so that means we need careers that allow us to track over time.  One-two year starters don’t help much (or at all).  Anyway, here are the QBs I included:

Peyton Manning, Eli Manning, Drew Brees, Matt Stafford, Philip Rivers, Tom Brady, Carson Palmer, Ben Roethlisberger, Tony Romo, Matt Schaub, Michael Vick, Matt Hasselbeck.

Then, I removed any season in which the player did not have at least 100 pass attempts.  For example, Tom Brady had an interception rate of 0 in 2008….because he only threw 11 passes before getting injured.

From there, I matched each player’s season interception rate with their rate the following season, ending up with 111 matched pairs.

The Result

Screen Shot 2014-01-20 at 10.14.09 AM

 

Hmm….not what Eagles fans wanted to see.  The correlation value is 0.12, so real but relatively weak.  In other words, a good interception rate one season was not very likely to result in a good interception rate the following season.  OR, interception rate is composed of some skill plus a fair amount of luck (that sounds about right).

I mentioned one issue with this analysis above (sample bias), but I want to mention another big one here.  We haven’t accounted for defensive strength.  It’s possible (likely in fact), that good defenses intercept passes at a higher rate than bad defenses.  Some of the variation in QB Interception rate is therefore explained by differences in the year-to-year schedule (which are largely random).

As I said, informative not dispositive.

A few more things

After collecting the data I looked at it from a few other angles, which led to a few interesting takeaways.

– Of the 13 QBs I looked at, the largest single season deviation from their overall average interception rate (NOT career because it’s not weighted) was 2.55%.  That was from Matt Stafford’s rookie year, when his interception rate was 5.3%.  The second highest deviation was 2.23%.  That was from Peyton Manning’s rookie year, when his interception rate was 4.9%

In fact, 4 of the 13 QBs recorded their highest seasonal interception rate in their rookie years.  Moreover, another 4 of them had rookie interception rates than ranked as their second worst season.  So together, 8 of the 13 QBs had either their worst or second worst interception rate their rookie seasons.

That doesn’t really TELL us anything, but it certainly suggests that QBs may improve their ability to avoid interceptions over time (which matches the “conventional wisdom”).  That, of course, would be great for Nick Foles, whose rookie rate was just 1.9%.

– In light of the last point, I thought it would be interesting to take a look at the progression of each QB’s seasonal interception rates.  Maybe from one year to the next there is a lot of variation, but over time QBs generally get better (or plateau around their “true” skill).  Here are some individual charts, pay close attention to the X-Axis label changes if you’re comparing:

Screen Shot 2014-01-20 at 10.35.14 AM

Screen Shot 2014-01-20 at 10.36.30 AM

Screen Shot 2014-01-20 at 10.37.40 AM

Screen Shot 2014-01-20 at 10.38.19 AM

Screen Shot 2014-01-20 at 10.39.00 AM

Wow…now that looks interesting.  Every one of them has seen a clear downtrend in interception rate from season to season.  Of course…it wouldn’t be a QB breakdown without Eli Manning:

Screen Shot 2014-01-20 at 10.35.50 AM

He really does ruin everything….(and he throws a LOT of interceptions; compare his X-Axis to the others).

– The key to remember, though, is that Nick Foles registered an interception rate of 1.9% in his rookie year and just 0.6% this year.  His career rate is now 1.2%.

That means, even if he is due for some regression, he’s got a lot of room to work with.  He could triple his career rate next season and still be at just 3.8%.  That’s high, but every QB in the sample except Rivers, Brady, and Rodgers, have hit that level at least once in their careers.

Additionally, if Nick Foles can IMPROVE his rate over time, as the QBs I showed above did, then he really will have an identifiable “elite” skill.  That’s probably unrealistic (you just can’t get much better), but remember that an improvement in skill would counteract the regular variance he’d expect to see.

– A lot more data to look at regarding Interception Rate, but for now I’d say the takeaway is this:

Nick Foles is very likely to throw interceptions at a higher rate next season than he did this year.  However, I wouldn’t bank on a massive shift, and given where his career rate is, I STILL expect him to finish the year with a very good interception rate (< 2.5%).

That’s good news for Eagles fans.

How often do good offenses score?

Were you underwhelmed by the Eagles’ offensive performance against the Saints?  Readers here might not have been, but my guess is a lot of fans were.  This was one of the best offenses all season yet it scored just 3 TDs on 11 drives (and a FG).  You can probably guess where I’m going now:

Context.

What does a good offensive performance really look like?  Let’s look at a few stats to find out.  In the process, I think we’ll find a better perspective with which to judge all teams offensively.

How many points per drive should a good offense average?  Take a guess.  If an offense scores a TD every drive, it’ll average 6-7 points per, depending on the conversation choices.

How about 2.98?  Does that sound reasonable?  Barely less than a FG.  Or 1 TD every 2.3 drives.  Clearly, that’s a pretty good offensive performance.  However, broken down like this, it’s hardly spectacular.  Of course, as you probably suspect by now, that was Denver’s average points per drive this season; Denver also scored more points this year than any team in NFL history. (FootballOutsiders.com)

The MEDIAN this year was 1.91 points per drive.  That’s 1 touchdown every 3.66 drives (with no FGs).  Or, that’s 2 FGs ever 3.14 drives.  Imagine starting a game and having your first 3 drives go thusly: Punt, FG, FG.   Are you satisfied?  Probably not.  However, that’s 2 points per drive, which would place 12th overall this year.

Punt, FG, FG…repeat.   Congratulations, you’re a top 12 scoring offense.

The moral of the story is, offenses don’t score nearly as often as many people think they do.  It’s frustrating to watch a series of punts, and we inevitably start questioning the play-calling in such situations.  However, it’s a lot harder to score than people think.

The Eagles averaged 2.18 points per drive this season, good for 8th highest in the league.

Against New Orleans (the 10th ranked defense by DVOA), the Eagles scored 24 points on 11 drives.  That’s 2.18 points per drive…. In other words, the Eagles offense, on a scoring per drive basis, performed EXACTLY as expected.  In fact, it performed better than expected when you account for the strength of schedule difference between the regular season and the playoffs.

Meanwhile, New Orleans averaged 2.4 points per drive this season, 3rd best in the league.   Against the Eagles, the Saints also had 11 real drives (excluding kneels).  They scored 26 points.  That’s 2.36 points per drive, ALSO roughly in line with their season average.  Given the low ranking of the Eagles defense, it’s also reasonable to say the Saints should have been expected to score MORE.  That’s not really the point of this post, but I found it interesting nonetheless.

Forget about this “per drive” crap, what about TDs?

Good question.  How many TDs do you think the average team scores each game?

The Denver Broncos scored 4.4 offensive TDs per game this year (teamrankings.com).   Of course, that was the best offensive performance ever (a huge outlier).  In fact, that measure was roughly 40% better than the second place team…..the Eagles with 3.2 per game.

The MEDIAN team this year scored just 2.4 touchdowns per game.

The average # of offensive drives this year was 186.  That’s 11.62 drives per game.  If we use the median value of 2.4 TDs per game (to keep Denver from skewing), that means, roughly speaking, the average offense this year scored a TD on 20.6% of it’s drives.

1 touchdown every FIVE drives.  That’s average.  

Like I said…perspective….context.  Even the best offense ever, Denver this year, scored TDs on just over 1/3 of its drives.

Remember that the next time somebody complains about the offense’s “inconsistency”.

Maximizing the Kick Return Game

Time for another guest post from Jared.  For those of you who don’t know, Jared is my brother.  He’s also a U of Chicago MBA and a two-time Jeopardy! champion.   You can follow him @jaredscohen or see the original of this post at his site, linked here.  Now, his words:

Last year I put together some analysis of NFL kick returns. I was really motivated by one big question – Why do teams return kicks?

Initially, I wondered if returning kicks was even the optimal decision for teams trying to win football games. I wondered if the risks of turnovers and poor field position meant teams really should prefer a touchback to bringing the ball out of the end zone.

As a brief review, that’s not the case. Returning kicks is, on average, better for scoring points than taking a knee in the end zone as the returns leave your team with better field position. If you look at it in terms of expected points generated on kick returns vs. generated on touchbacks – the distinction is clear: (Note: this analysis relies on the concept of expected points based on field position – which I’ll assume readers have already seen and grasped)

This data comes from the first 16 weeks of this NFL season, over 2400 kicks. It’s also consistent with last year’s data.

So returning kicks is good, but think about why it’s a good idea. Although it presents better average field position, the average return nets only about four yards of position (and only two yards if the ball is brought out of the end zone relative to a risk-free touchback).

Linking back to material my brother has posted – the upside is directly tied to variance. Returning kicks is much more of a high-variance strategy.

Below – is an illustration of all returned kicks through Week 16 this year. The histogram shows the distribution of expected points.

You see the giant spike between 0.3-.04 which equates to a return between the 18 and 22 yard lines, that’s the most typical result (remember a touchback is worth 0.34 expected points). But there’s also an extremely long tail of positive performance, and these outliers can be worth a lot more (even a touchdown). Those outliers are what make kick returns worth the risks (injury, turnover), which is exactly what we mean when we talk about high-variance strategies.

A touchback has zero variance. That result is predictable and constant. But a return, that could be a whole bunch of possibilities.

OK – so let’s take the idea that returning kicks instead of taking touchbacks is a high-variance strategy as a hypothesis. Now, if that’s true, we would expect to see a couple different trends in the data. Generally, we would expect less talented teams to return kicks MORE often than their better opponents. Weaker teams should be pursuing higher variance plays in an attempt to pick up ground on those other (stronger) squads. In an example – you’d expect the Jaguars to try everything to beat the Broncos because Denver is extremely talented and playing a conventional game will leave the Jaguars at a big disadvantage. That could mean any number of things, more shots downfield, 4th down conversion attempts, surprise onside kicks, and we could expect – more kick returns.

So…is that something we actually observe in the data? Are weaker teams pursuing higher variance strategies in the form of more frequent kicks?

To test this, I went back and looked at my favorite kickoff metric – percentage of touchback eligible kicks returned. This counts the number of kicks that were returned out of the end zone as a proportion of the total number of kicks fielded in the end zone. Obviously – teams will return all kicks fielded short of the end zone, so we need to exclude these. The real decision point is whether or not teams bring balls out of the end zone – this is our true high variance strategic choice.

The data set it built off of play-by-play information, which is the best I can get. Unfortunately, there are a large number of touchback kicks where distance is not recorded and it isn’t specified whether the kick was fielded or kicked out of the end zone. After some initial eyeballing I’m confident these are kicks out the back of the end zone (Matt Prater of the Broncos had a lot of them as an example). So our set of kicks is a little smaller than you might expect. But there are still 950 kicks in our sample.

Then, I took all the NFL teams and split them into three performance tiers based on point differential. Teams with the highest point differential are members of the first tier, teams with the worst scoring differential are in the third tier. Below are the teams and their tier positions.

You can see the usual suspects in both the first and third tiers. And to me, this is where we’d expect to see the biggest change. These third tier teams – they have to do MORE to compete against first tier teams. Alternatively, first tier teams, one might argue, don’t need to take additional risk by sending their return man out of the end zone. If we look at touchback eligible kickoff return percentage across the different matchups – we can see if there’s any difference in the way teams behave. Do third tier teams return more kicks when they face off against first tier teams? Do first tier teams (who don’t need to pursue high-variance strategies) return fewer kicks?

Hmm…there’s almost no difference in return % whether the worst teams are facing other crappy teams or the best teams. That seems a little odd…as we had guessed the worse teams SHOULD be returning more kicks when they face better teams. This indicates that this doesn’t happen.

It’s also not a result of sample size, as most of these cells are large enough (80-120 observations).

As another check, I looked at touchback eligible return percentage relative to specific team talent (via point differential) on a team-by-team basis. I did this to see if there were any teams that really seemed to be demonstrating aggressive tactics at the individual level.

Again, this doesn’t appear to support our thinking that poor teams are pursuing higher variance strategies by returning more kicks. At best, it’s inconclusive. There are a couple of teams, like the Vikings, who really push the envelope – but there’s not a major correlation between team talent and return percentage (correlation is roughly -0.15)

Strange, but maybe identifying high-variance strategies before the game starts and following them blindly isn’t really what coaches of less-talented teams spend time on. Is there another way we can test our hypothesis?

Another theory is that if teams aren’t determining to return more kicks as part of pre-game strategy, maybe it’s something they pursue once they fall behind on the scoreboard. This wouldn’t even have to be exclusive to poor performing teams – any team that’s fallen behind might be more likely to run back kicks to try to break a big play to help catch up. What if we examine touchback eligible return percentage by in-game score differential?

The chart below illustrates the return percentage across a set of different score bands, ranging from down by more than 14 points to ahead by more than 14 points.

Again – there doesn’t seem to be any real connection between the scoreboard and aggressive kick return tactics. A team down by more than two touchdowns is just as likely to return a kick out of the end zone as one who is tied. If a kick return out of the end zone is indeed an aggressive play with a higher reward – teams don’t appear to be pursuing it MORE when they need to make up ground or LESS when they have a large lead. (As an aside, I absolutely cannot explain why having a small lead seems connected to a dramatic drop off in returns. I’ll chalk that up to some data wonkiness unless someone has a great insight there.)

But the broader concern remains. Shouldn’t teams which are behind or less talented need to take more chances to win? Why aren’t they doing that and bringing kicks out of the end zone?

My initial guess, though I’d welcome other speculation, is that teams the organizational structure of coaching almost inhibits something like that from happening. This comes with the obvious caveat that I’ve never coached in the NFL (so sure, Bill Belichick or someone else can dismiss all this out of hand as mom’s basement musings – but screw them). But if you’re the special teams coach of an NFL team – your work includes a thorough evaluation of your special teams and your upcoming opponent. All that work and planning becomes a little less valuable if a head coach just says – ‘Hey, I think we should return any kick we get in the end zone’

If the special teams coach is to maintain any kind of control over what his squad does – a simplistic rule like ‘run them out when we’re behind’ may not be sophisticated enough to justify all that pre-work and planning.

But that’s just a thought, based on the idea that coaches know their teams and customize approaches based on their own teams’ skills and the matchup with the opponent. Of course, when you actually look at the data, teams don’t really appear to be all that successful in managing their return game. Below is an illustration of touchback eligible return percentage, but this time charted against the average return position (i.e., return ability).

While we’d expect to see some correlation here – to show that teams with good return games return more kicks and teams with poor return teams take more touchbacks – that’s only true to the degree of a 0.2 correlation.

Some teams seem to get it – the Bills are really bad in the return game, but they rarely return kicks out of the end zone (on a relative basis – still over 50%). At the other extreme are the Vikings. The have Cordarrelle Patterson and, as such, they return kicks out of the end zone over 95% of the time!!!

On the flip side, look at Washington and St. Louis, teams with mediocre return units that run kicks out of the end zone 90% of the time. The Chiefs and Ravens seem odd as well – teams with great performance who could stand to run some more back. Now, maybe the Redskins are pursuing a high variance strategy, and maybe the Chiefs a more conservative one, but the overall results remain inconclusive.

At the end of the day, I come back to the idea of coaches and control over their special teams. For any team to read any of this and think about employing a ‘high-variance’ strategy – it really requires an admission of the role of chance in the outcome of a football game. Running every kick out of the end zone is a strategy based on the concept of inherent variability in outcome. Some returns may get stuffed, and others may go for big returns, but you can’t be sure when one or the other will happen. That view, to me, is fundamentally opposite the idea that with the right scheme and flawless execution – you can create the optimal outcome.

One of those ways of thinking supports the coach as the ultimate authority, while the other incorporates more probabilistic thinking. That gap is why I think we haven’t seen any patterns to support our hypothesis, and no clear evidence of high-variance kick return strategy consistently employed in today’s game.

Two-Factor Blitz Theory

I received some pushback from yesterday’s Billy Davis rant, so today I’m going to try to add some nuance to my explanation.  First, I want to note that while Davis bears the brunt of my criticism, he’s certainly not the only DC I disagree with on a fairly consistent basis.  Graded against everyone else, Davis is OK (for now).  However, as is usually the case, just because everyone else does something doesn’t mean we need to do the same thing.  Conventional wisdom, especially in sports, frequently lags the “optimal” strategy.

So….Defensive Strategy, and more specifically, the Blitz.

My general take on this is that the Blitz (sending more than 4 pass rushers) should be viewed as a TOOL, not a general philosophy.  I realize that in Philadelphia, that’s borderline heresy (lot of Jim Johnson fans out there).  But let me explain.

Two Factors

To complete a pass, two things must happen (generally speaking): an “open” receiver must exist, and the QB has to identify that opening (after which he presumably throws the ball there).  It’s tough to determine what constitutes an “open receiver”, so I’m going to discuss this side of things in terms of Windows.  So a passing window refers to an opportunity to place the ball where the receiver can catch, and one must exist and be identified in order to complete a pass.  Simple enough?

Also, for a QB to identify the available passing window, he must have TIME to do so.  The more time he has, the higher the odds of him seeing an existing window or of one developing.

By breaking the process down into these factors, we can see the basic trade-off in defensive strategies (against the pass).  The best of both worlds, of course, is to minimize the passing windows AND minimize the time the QB has to identify them.  That’s why DEs are so coveted.  If you can generate a strong rush (i.e. lower QB time) with just 4 d-linemen, you can use everyone else to close passing windows.  However, very few teams area able to do that on a regular basis.

More often, you have to make a choice.  You can rush an extra man (blitz), which should decrease the amount of time the QB has to see a window.  Conversely, you can rush fewer men, and use more of them to minimize the windows.

With me so far?  Good, now let’s talk a little bit about passing windows.

Passing Windows

Passing windows open and close throughout each play.  A complete pass occurs when one of them opens and the QB hits it.  To help illustrate, I’ll pick a random frame from Sunday’s game:

Screen Shot 2013-12-31 at 9.57.59 AM

Nice….Now let’s superimpose the passing window on it.  Despite the outcome of the play (Boykin game-saving interception), there was, in fact, a window to hit here for Orton.

Screen Shot 2013-12-31 at 10.02.04 AM

That’s a rough approximation, obviously, but you can see the idea.  Given such a big opening, how the hell did Orton miss?  Any guesses?

How about:  He’s not a good QB!?

That’s a little unfair (just a little), because every QB misses opportunities sometimes.  However, let’s dig a little deeper into this.

When deciding what pass-rush strategy to use, there’s perhaps no greater factor for consideration than the skill of the opposing QB.   We now have to combine the QB skill with our Window illustration from above.  Let’s visualize it like this:

Screen Shot 2013-12-31 at 10.11.23 AM

Don’t get too caught up in the relative sizes, this is far from a “to-scale” illustration.  In the middle we have the passing window.  On either side I’ve provided a visual representation of each QB’s (Peyton and Orton) accuracy.  Think of the two layers as confidence intervals; something like 70% certainty the ball will end up within the smaller red circle and 90% certainty it will end up within the outer circle’s boundaries.

Hopefully this is rather intuitive.  Now play a mental game using those images.  The green square will move across the screen from left to right.  You control the red circle, and your job is to align it with the passing window and and press go.  Think of it like aiming a rifle.

Now…which player’s range (red circles) would you rather play with?

Easy, Peyton Manning’s, because his confidence ranges are smaller, meaning there’s a smaller margin of error.  For example, let’s say you align each perfectly with the passing window:

Screen Shot 2013-12-31 at 10.19.15 AM

See the problem?  The window is smaller than Kyle Orton’s accuracy range.  Meanwhile, it the window is significantly larger than Manning’s 70% accuracy range.  The upshot, naturally, is that Manning is a lot more likely to complete this pass.  Going back to our game image above, we can visualize the pass like this, with the yellow X denoting the final placement of the ball, which was intercepted.

Screen Shot 2013-12-31 at 10.23.35 AMMoreover, we can extrapolate the idea.  In general, Peyton Manning will be able to hit smaller windows than Kyle Orton will.  Obviously, smaller windows occur more frequently than larger windows, hence Peyton Manning, by virtue of his accuracy, will have many more opportunities to complete passes than Kyle Orton.

Now pretend you’re a defensive coordinator.  Remember you have a choice to make between eliminating passing windows and minimizing time.  In this exercise, you cannot do both.  Against which player is the “window elimination” strategy more likely to work?

Easy (again), Kyle Orton.

Since Peyton Manning’s required window size is so small, eliminating them will require extremely good coverage.  More likely, you can play excellent coverage on the receivers and STILL not prevent several of these small windows from opening up.

Conversely, against Kyle Orton, things aren’t so difficult.  He needs a relatively large window.  Large windows are easier to eliminate.  You don’t have to play perfect coverage. Notice in the Boykin play, there was a relatively large are in which Orton could have complete a pass for a big gain.  Fortunately for the Eagles, Orton didn’t hit his spot.  Boykin’s coverage was far from perfect, but it didn’t need to be!

In light of that, go back you your strategy decision.  Which do you think is easier to do:

– Eliminate windows

– Minimize time

Now consider that Orton was operating out of 3-step drops for much of the night.  Then, the answer is easy.  Eliminating his passing windows is the much higher-probability play.  Note that’s the BASE strategy.  Obviously, you need to blitz every once in a while, if for no other reason than to add some unpredictability.

That’s the crux of my argument against Billy Davis’ blitzes.  He doesn’t seem to vary his usage as much as I believe he should, and he doesn’t save his blitzes for high-leverage situations.  Instead, he uses them in LOW-leverage situations, where the reward of a sack is comparatively low, especially when weighed against the odds of a big play.

Against a great QB (like Drew Brees this week), you have to be much more aggressive because it’s much more difficult to eliminate those passing windows.  Moreover, there’s another factor to discuss:

The Blitz Bonus

A very successful blitz will result in a sack.  A sack dramatically swings the odds of a turnover (punts included) in the defense’s favor.  Now, comparing opposing QBs, against which ones do you think that’s most important to do?  I’ll give you a hint, it’s not Kyle Orton.

Against a great offense or great QB (frequently one and the same), the odds of allowing a 1st down are comparatively high.  For example, according to Pro-football-reference.com, the Denver Broncos faced 93 third downs needing 5 or fewer yards for a first down.  They converted 62.4% of those.

Now compare that to a bad offense, like Baltimore (ranked 30th by Football Outsiders).  The Ravens faced 96 third downs with 5 or fewer yards to gain.  They converted just 49% of those.

As you can see, getting to 3rd and less than 5, normally not considered much of a “win” for the defense, is still good enough to get you to 50/50 against a bad offense.  Assuming each opportunity is an independent event, the odds of the Ravens converting two consecutive such third downs is just 25%.

Hopefully your mental light-bulb just turned on.  Facing a Kyle Orton-driven offense, the Eagles were looking at a team much closer to the Ravens than the Broncos.  In that situation, just preventing a big play and forcing the Cowboys to convert a string of third downs was VERY LIKELY to produce a punt.

In other words, we didn’t NEED a sack!  The odds were already in our favor.  Conversely, if we had been facing the Broncos, the risk/reward equation flips.  That team is much more likely to convert a string of third downs, meaning the defense needs to do something to increase its odds.  Getting a sack is one of the only affirmative ways to do this.  In that case, the reward of getting a sack outweighs the risk of giving up a big play.  Without the sack, you’re likely to give up a long drive anyway!

Against a bad offense, though, that’s not the case.  It’s better to sit back, eliminate passing windows, and wait for the odds to shake themselves out.  By blitzing bad QBs, you’re making a foolish grab for upside that you don’t need.  Bad QBs will struggle to hit receivers that are even marginally covered, so why make it easy for them by making those passing windows larger?

Wrapping Up

Hopefully that illuminated things a bit more clearly.   Basically, against bad QB’s, the odds are already in your favor.  The reward of a sack (or forced incompletion), and the increased odds of a punt that come with it, are NOT worth the risk of the big play. In all likelihood, a bad offense will NEED a big play in order to score.  They simply won’t be able to string together a 12 play drive with a lot of 3rd down conversions.  Hence, the goal should be to get to third down as often as possible, and let the odds take effect.

Against a great QB, though, that’s not enough.  They ARE somewhat likely to string together 3rd down conversions, especially if their short yardage situations.  Similarly, they DO NOT need a big play to sustain a drive.  In that case, the risk of giving up the big play (which is worth relatively less to a great offense than to a bad one) is worth the associated reward of a longer yardage situation (which the defense needs to push the odds in its favor).

That doesn’t mean you never blitz a bad QB or always blitz a great one.  It does mean that you’re general pass rush strategy, particularly when it comes to sending extra pass rushers, should vary greatly depending on which QB you’re playing against.  Just saying “we’re a blitzing defense”, so we’ll blitz, is a very low-level strategy.  It’s far too simplistic, and sounds a lot more like a crutch than a well-thought out, adaptable and deployable strategy.

The Benefits of Being a High-Variance Team

Great game yesterday.  It was a nice preview of what this team COULD be if both the offense and defense play well at the same time.  Can’t ask for a much better set-up for the Eagles than a win-or-go-home game next week in Dallas.  The Eagles are, objectively, a much better team.  The stakes should take care of the motivation aspect.  Also, with Romo being out (assuming the news is accurate), if Foles shows up looking anything like “GoodFoles”, there’s very little chance of Kyle Orton keeping pace.

Now, to today’s topic.

According to Football Outsiders, the Eagles are ranked 31st in the league by DVOA Variance, at 25.4% (BEFORE the Chicago game).

Only St. Louis has been more uneven.  Normally, you’d prefer your team to be both very good, and very consistent (low-variance).  That’s the goal.  However, there’s more to the story, and it ties in to our general underdog strategy discussion.

The Eagles are not the best team in the NFC.  They might not even be in the top 5 (before yesterday, DVOA had them 7th in the NFC).  That means that winning the Super Bowl will require winning multiple times against inarguably “better” teams.  When I say better, I mean the expected performance of the other team is clearly higher than the expected performance for the Eagles.  Of course, that’s only one part of the equation.  The other, obviously, is variance.

The fact that teams don’t always perform to expectations is exactly what makes the game fun.  Otherwise, there’d never be any upsets.  So…taking the next step, that means if you’re a large underdog, you really want at least one of the teams involved (you or the opposition) to be a high-variance team.  Remember, underdogs (both ex-ante and as a result of current conditions) want to MAXIMIZE variance.

Let’s illustrate.  Below is a graphic showing the expected performance distributions for two teams.  Unfortunately, the shape-options in Powerpoint are fairly limited, so the shapes are a bit crude.

Screen Shot 2013-12-23 at 7.23.36 PM

Above, the width of each distribution and it’s height at each point tells you both how good the team is and how consistent it is.  If this were a “to-scale” drawing, the area under the curve would add up to 1.  Notice that in the above chart, there is a gap between the two teams.  That means, in this case, the Red team would NEVER beat the blue team.

Let’s pretend you’re the Red team.  What can you do?  Obviously, you can’t do anything to Blue’s distribution.  In general, the whole point of team-construction is to move the distribution to the right, so that’s option A.  If you shift Red far enough to the right, you’ll catch up to Blue.

But what if it’s in-season?  What if you only have one week before the game?  You can’t do much to change the make-up of your team, so Option A is out.  There’s still hope, though.  You can WIDEN the performance distribution.  This is what it means when we say  teams in desperate situation must make High-Variance moves.  Let’s say Red team had the same average performance expectation, but is now a High-Variance team.  Then the chart might look something like this:

Screen Shot 2013-12-23 at 7.31.55 PM

See the overlap?  That’s the key.  Although it’s still unlikely, there is now an actual possibility of Red beating Blue.  Notice that there’s also a possibility of Red losing by a lot more than it would have before.  However a loss counts the same whether it’s by 1 point or 40 points. (Ask any real racer….)

Back to real life: the Eagles have a very WIDE expected performance distribution; it’s reflected in their high-variance.  That means that even if they’re undeniably worse, on average, than a team like Seattle, they’ve still got a decent shot at winning (compared to if they were a low-variance team.)

For example,

Prior to yesterday, the Arizona Cardinals ranked one spot ahead of the Eagles by DVOA (10.9% to 7.7%).    However, the Cardinals are among the most consistent teams in the league, and rank 4th overall by Variance, ahead of the Eagles by 27 spots, and a variance margin of 18.9%.

Charting each team against Seattle (very rough approximations here), we’d get something that looks like the following:

Screen Shot 2013-12-23 at 7.55.05 PM

Notice that while the Eagles’ average is worse than the Cardinals, their overlap with Seattle is actually greater than the Cardinals.  We’d have to do some calculus (and put a lot more effort into an accurate chart) in order to calculate the difference, but the overall idea is sound.

While it’s incredibly unlikely, the Eagles do stand a greater chance than similarly skilled teams to actually win the Super Bowl if they get to the playoffs by virtue of their high-variance nature.

Lastly, let’s look at the regular season variance of recent Super Bowl winners (this is going to warrant a dedicated post, but let’s just take a peak for now):

Baltimore Ravens – 15.6%, 24th in the league

New York Giants – 15%, 20th in the league

Green Bay Packers – 14.8%, 15th in the league

New Orleans Saints – 15.8%, 17th in the league

Pittsburgh Steelers – 10.8%, 8th in the league

Mean Reversion and the 2012-2013 Eagles Improvement

Preseason, I did a number of posts that focused on the reasons why the Eagles finished with such a poor record last year.  The general thesis was that the team was bad, but it was also very unlucky.  Therefore, we could expect a better record this year purely as a result of reverting to the mean in several meaningful statistics.  Today, let’s take a look at a couple of them and see how they look.

First, here’s the 2012 performance dashboard I put together.

Remember that I scaled everything by historical standard deviation (last 10 years of data) so that it could all be viewed in one chart.  For our purposes today, the most important terms above are Fumble Recovery %, Fumbles Lost, and Net Field Position.

Note that, for now at least, I’m going to avoid the whole luck-vs.-skill angle.  I’ve explored that before and I’m sure I’ll revisit it again.  Regardless of which side you believe in, the fact is that regardless of the role of luck, those statistics show NO PERSISTENCE from year to year.  Note also that the three stats I’m highlighting are obviously interrelated, so it’s no surprise that terrible performance in one is correlated with terrible performance in the others.

Fumble Recovery %

In general, teams should expect to recover around 50% of all fumbles.  There’s been some additional research done about varying rates for different TYPES of fumbles (Downfield WR vs QB for example), but after including all types, the overall rates converge to 50%.

Last year, the Eagles recovered just 35.09% (Teamrankings.com), which is 1.99 standard deviations below the mean.  That’s really bad, and extremely unlikely to happen again.  So how is the team doing this year?

46.34%

Not great, but a much more reasonable rate of recovery.

Fumbles Lost

Relatedly, the Eagles problem last year wasn’t just the rate of recovery, it was an overwhelming number of fumbles.  Combined, that meant the 2012 Eagles lost a historically large number of fumbles to the other team.  Looking at the chart above, we see that the team lost 22 fumbles last year, which is nearly 3 standard deviations from the mean.  Like I said, historic, and a big reason why last year’s team struggled so much.

So how do things look now?

Well so far, the team has lost just 8 fumbles, or .615 per game, meaning it’s on pace for just under 10 fumbles lost, less than half of last year’s measure.

Net Field Position

Finally, for today at least, there’s Net Field Position.  As a result of both special teams and the historic turnover rates, the 2012 Eagles had TERRIBLE net starting field position.  Looking at the chart above, we see the team’s average drive started 6.67 yards behind the other team’s average starting position.  That’s a very big difference, and it’s more than 2 standard deviations from the mean.  The offense last year was actually middle-of-the-pack by yards-per-drive.  The problem was that they had farther to go than everyone else.

This year?  +1.4 yards, good enough for 11th overall (Football Outsiders).

Having trouble conceptualizing the significance of the shift?  Well consider this:

This year, the team is averaging 33.06 yards per drive.  It’s scoring 25.7 points per game.

Last year, the team averaged 31.51 yards per drive.  It scored just 17.5 points per game.

Put differently, this year’s team is gaining an average of just 1.5 yards per drive more than last year’s team did.  

The real difference?  Mostly turnovers and field position, both of which we’re primed for mean reversion.

Lastly, the really good news

Did you notice anything else about the stats I just discussed?  Let’s look at them again:

Fumble Recovery %: 46.34%

Fumbles Lost:  On pace for 10

Net Field Position:  +1.4 yards (11th overall)

Now?  While last year’s numbers were EXTREMELY bad, and thus carried a very high probability for improvement, this year’s numbers are squarely in the middle of the expected range.  That means, while last year’s team was both bad AND unlucky, this year’s team is just good, no luck caveat needed, at least as it pertains to these stats.

That means what we’re seeing isn’t likely to be a fluke.  Once the season is finished I’ll look at a larger number of statistics and see where we can expect improvement or decline, but for now, it looks like the team is just good.

P.S. I’m in the middle of the law school exam period, hence the low volume of posts.  Good news is I’m finished next week, meaning my break coincides with the home stretch of the season, and I’ll be able to post a lot more frequently, at least until late January.

QB Performance Frequency Distributions

Ok, so a couple of weeks ago I posted about how often great QBs have bad performances. Today, let’s take a more detailed look at things.  The overall question is, are quarterbacks equally likely to outperform or underperform their long-term average QB rating in any individual game?  Stated differently, are the individual performance distributions symmetrical?  Are they Normal?

Let’s start at the top.  Here’s the graph for Peyton Manning.  Note that the X-Axis labels show the UPPER Bound of each bar.  So the “60” label means that bar corresponds to games where the player’s rating was between 50 and 60.  Also, this is all games with at least 10 pass attempts.  Remember that these are NOT weighted numbers, so they’ll be different from the career measures for each player.  This helps to minimize the skew effects of games with a lot of pass attempts (garbage time yards in a blowout loss for example) as well as increase the weight of great games with relatively few attempts (when a team has a big lead early perhaps).  Realistically, we just want to know what level of performance we’re likely to get in the NEXT SINGLE game.

Screen Shot 2013-11-27 at 4.26.10 PM

That looks pretty Normal.  The Mean is 97.45 (note that this is NOT his long-term average, since it’s not weighted by attempts).  The Median is 95.6.  Obviously, those are crazy-good numbers.  That’s why he’s a HOFer.

Big-picture, if players generally follow a Normal distribution, then we can tell a lot about Nick Foles from relatively fewer games.  So Peyton’s chart is really encouraging.

But here’s Drew Brees:

Screen Shot 2013-11-29 at 10.54.36 AM

Not nearly as neat as Peyton’s.  The Mean is 95.17, the median is 92.4, and there’s some clear skew to the distribution.  Going back to the last post (linked above), notice that Brees has had more games with ratings between 60-80 than he has games with ratings between 80-100.  Overall, his performance, though still amazing, is less predictable than Manning’s.  The standard deviation of Brees’ game log is 29 (rounded) while Manning’s is 27.  Illustrated differently, we can look at the range covering the middle 50% of performances:

Manning:  79.9-112.2

Brees:  72.8-116.7

Again, both great, but Brees is less predictable.  We could raise some interesting strategic questions here as far as which one you want in which situation, but I’ll save that for another day.  For now, just imagine tying it back to our “David Equation” (that’s what I’m calling it now).  Brees might not be as good, but his higher-variance play might be preferable for an underdog team, while you’d rather have Manning if you’re the favorite.

Now let’s take a step down and look at some non-future-HOFers.

Here’s Sam Bradford:

Screen Shot 2013-11-29 at 11.04.18 AM

Much uglier, as expected.  The mean is 80.48, the median is 81.2.  The standard deviation is actually much lower than either Manning or Brees, at just 21. His middle-50% range is:

66.3 – 91.3.

That’s what a bad QB looks like.  Now we should probably caveat all of this by saying it’s a bit unfair to evaluate the QBs in a vacuum, with no regard to the talent level they’re working with.  That’s the case with just about every NFL evaluation, it’s just the nature of the game.  Still, Bradford hasn’t been good enough, and I’m skeptical he ever will be…

How about Eli? (You knew I couldn’t leave him out)

Screen Shot 2013-11-29 at 11.41.26 AM

 

Mean of 82, Median of 81.65.  Standard deviation of 27.3.  Hmmm…those numbers look vaguely familiar.  What were Sam Bradford’s again?  (mean of 80.48, median of 81.2, stdev of 21).

Interesting.  Future HOFer Eli Manning’s average a median performance are almost identical to secret-bust Sam Bradford (secret because nobody seems willing to say it outright.  I will, Bradford is a bust, unequivocally.  Closing in on 2000 pass attempts, he has a com % below 60 and a Rating below 80).

What about Eli’s middle-50 range?  63.6 – 100.7.

That’s just about the definition of mediocre (and maybe even a bit worse, we’ll see later).

Before I move on, let me repeat one thing.

Eli Manning’s MEDIAN performance is a rating of 81.65.  So HALF of his starts are WORSE than that.

Going back to the initial question I posed:  Are individual QB performance distributions symmetrical?  Almost, though we have to grade it as inconclusive since I only looked at a few QBs.  So we might be able to use that to infer some info about Foles.  Clearly, though, they’re not Normal… There’s a lot more we can do with this type of data, but I’m going to have to wait for another day to start on it.

Also, as is usually the case, I think I stumbled onto something more interesting (the middle-50 ranges).  So rather than go through each QB and post a chart of they’re distributions, I’m going to end this post now and start making a table of every starting QB’s Middle-50 range.  Then we can start talking about which is “best” in a given situation, with some real data to go from.

Before I go, here’s Nick Foles.  He has just 11 qualifying games, so small sample is an understatement, but it’s fun to look anyway.

Screen Shot 2013-11-29 at 11.59.26 AM

 

Average is 97.2, median is 96.6.  Standard deviation is 35.8.

And here’s Mike Vick.

Screen Shot 2013-11-29 at 12.02.51 PM

 

Average is 80.9, Median is 83.6.  That’s 2 points BETTER than Eli’s median performance…. Vick’s standard deviation is 28.6.

Happy Thanksgiving