Hacking the Draft

For those of you who don’t remember, last season I did a historical analysis of draft pick success according to position and round.  Below is one of the posts stemming from that project.  At the bottom are two charts showing round-by-round probabilities.   Note: counting someone as a “starter” is a bit subjective.  Inclusion in the below data means that a player started for at least 5 years in the league (according to Pro-Football-Reference), or if they joined the league less than 5 years ago, has started for more than half the time.  The sample is all players drafted between 1999 and 2011.

Hopefully everyone has enjoyed the round-by-round breakdown.  While there are obviously a number of variables that can’t be controlled for, the pure statistical look at each position group has already provided some interesting insight.

Before I get to the big chart, let me just clarify exactly what I think this type of analysis is good for, then dig into an Eagles example.  Feel free to skip to the chart and come back.

The purpose of this is NOT to arrive at a set of rules by which teams should draft players.  Instead, it is meant to provide a general guideline, or a “default draft position”.  For example, yesterday I said that selecting DEs in the 3rd round is a terrible value proposition according to the data I’ve collected.  That doesn’t mean drafting a DE in the 3rd round is ALWAYS a terrible decision, it just means that for a team to make that decision, it must see or know something about the subject player that makes him CLEARLY much better than any other prospect available at that position.

The biggest flaw in NFL draft strategy, as far as I can tell, is each team’s confidence in its own ability to evaluate talent.  Regardless of the general manager, every team has historically had a very large margin of error when it comes to talent evaluation.  For instance, in the 1st round less than 6 in 10 LBs selected from 1999-2011 had or are having significantly productive careers (according to our definition.)

So what does that mean?  It means teams, in general, should be mostly focused on value during the draft, as opposed to parsing prospects.  This is perhaps never more apparent than when a team decides to trade up to select someone.  Let’s use the Eagles as an example.  NOTE: This is a very rough example, with numbers pulled from my ass, and is only meant to illustrate a larger point.

When the team traded up for Brandon Graham, it swapped 1st round picks with Denver and gave up two 3rd rounders as well (moving up from 24 to 13).

At the 13th pick, no DEs had been taken.  Between picks 13 and 24, 3 were taken, including Graham.  So that means the Eagles, in their analysis, decided that they had to take a DE (will not argue that decision here, though I was mad they passed on Earl Thomas, and can produce witnesses that will verify I said that when it happened).

The only way the trade made sense was if the Eagles, in their DE analysis, decided that the odds of Graham becoming a stud DE were MUCH higher than the odds of JPP, Morgan, or Hughes becoming a stud DE.  Here is where the “margin of evaluation error” comes into play.

Using our historical draft data, we can calculate the odds of getting a starting DE with a 1st round pick and two 3rd round picks (I realize they were hoping more than an average starter, but stay with me).  Using the table below, we can calculate those odds to be 81.5%.  Using the Pro Bowl percentages from the earlier tables, we arrive at 37% for the odds of getting a Pro Bowl DE if you select DEs with a 1st round pick and 2 third round picks.  So here is the breakdown for “generic DEs”:

– 1st Round Pick – 24% chance of Pro Bowl, 67% chance of starting

– 1st Round Pick and 2 third round picks – 37% Pro Bowl, 81.5% starting.

Please note that this DOES NOT mean the Eagles made a bad decision.  Obviously the odds should be better for the 13th pick than for the 24th pick (we’ll get to that another time).  It DOES MEAN, however, that the Eagles, in their evaluation of Brandon Graham, should have been almost certain that he was more than 37% likely to be Pro-Bowl caliber, and more than 81.5% likely to start.

I have no doubt that they believed this, BUT, if they had applied a margin of error to their own analysis (as any good team should) prior to making the trade, they would have been unlikely to go through with it.  Let’s be extremely generous and assume the Eagles front office could peg these odds with a MOE of +/- 15% (WARNING: overly simplified statistics).   That means if they estimated Graham had a 90% chance of starting, his true odds of starting were almost definitely between 75-100%.

Immediately we can see an issue.  Even giving Graham incredibly high odds of starting (90%, which is more bullish than any team should be with any players outside the top few picks) and the Eagles a very generous MOE (+/-15%), the resulting range still does not exclude the 81.5% starting odds for the generic position of a first and two thirds (although it is near the bottom of that range).

That means the Eagles really should NOT have been (though I’m sure they were) confident that Graham would be better than just taking whichever DEs were available at #24 and in the third round, and hence, should not have made the trade.

I don’t mean to suggest that trading up is never a good idea, simply that the evaluative bar for whichever prospect is the target must be EXTREMELY high, and much higher than the standard currently being applied by most teams.

My final point:  Teams do not appear to take a probabilistic approach to drafting (which they almost certainly should), and I would guess that they do not actively overlay a margin of error onto their evaluations.  This is very much a “new-school vs. old-school” issue, similar to the statistical revolution in baseball, but IT IS NOT THE SAME.  It is largely a matter of GMs being willing to recognize and account for their own shortcomings and cognitive biases.  The franchises that can apply this will, in the long run, be more successful than those that don’t.  (Looking into which teams might be using this type of strategy will be another day’s post)

Ok, enough talk.  Here is the chart with every round included. I removed the All-Pro and Pro Bowl columns to make it easier to compare.  Dig into it and see what you think.  Later this week we’ll mine it for an “optimal default strategy”.

I updated the chart soon after the original post,  here is the updated version; so the numbers might not match the post exactly.  The original is reproduced below.

Screen Shot 2014-05-07 at 10.29.30 AM

Screen Shot 2013-01-29 at 10.54.06 AM


Examining the WR prospect tiers

There’s a lot of talk about the Eagles potentially trading up for a WR, or at least taking one with their 1st round pick.  Peter King has them giving up their 1st and 2nd round pick to get Odell Beckham Jr.  I’ve been very clear about why I think this is a poor strategy (trading UP for a WR is an unbelievably bad decision).  For more on those reasons, see my last post.  Today, though, I wanted to look at it from a different angle and discuss things in light of what the actual WR class looks like.  Previously, I left it at “it’s deep”, which doesn’t really provide the full context.

From my TPR rankings, here are the top 10 WRs in the draft.  Remember, since each of these guys play the same position and I took individual standard deviation out of the formula, these relative rankings are purely rankings are primarily the result of the NFL.com, ESPN, and NFP grades.  The multiplier stratifies the class a bit, but the effect is small.

Screen Shot 2014-05-06 at 10.04.32 AM

We can ignore Sammy Watkins and Mike Evans.  I’m sure Chip would love to grab Evans, but it’s extremely unlikely he falls out of the first 10 picks.

Instead, let’s focus on the second tier, the yellow shaded area.  Here we have four WRs that have all, at various points in time, been linked to the Eagles.  If the Eagles come out of the 1st round with a WR, it’s nearly a certainty that it’s one of these four guys.  But that’s not what we’re here for, is it?

Looks closely at each of those prospects and look at the grade assigned.  Now, how certain are you that you can identify which one will be the best NFL player?   “Not at all” is the correct answer.  I’m sure the Eagles have different grades and a different order of players, but the fundamentals are the same.  You need to ask yourself two questions:  How big is the difference between each prospect’s grade?  How big is the margin of error in our evaluations?

Within each tier, the MOE (if you’re being honest) is almost definitely larger than the difference in grades.  Therefore, practically speaking, they all have the same grade.  I other words, they all have the same expected value.

So why would you want to pay more for one of them than they other?

Now, let’s take aim at Peter King’s rumor, which is:

Eagles trade up for the 15th pick and select Beckham.

First, let’s see how necessary that trade is.  If they complete it, obviously they get Beckham.  If not, though, how likely is he to be available at the 22nd pick?

Well it just so happens that Brian Burke of advancedfootballanalytics.com (new name) has just released a Bayesian prediction model for the draft.  Obviously, we can’t put too much weight into this just yet, but it’s a very good representation of the type of thinking every team should be doing.  Here is Beckham’s output:

Screen Shot 2014-05-06 at 10.25.12 AM

According to this model, there is just a 7% chance that Beckham is available at the 22nd pick.  There is a 53% chance he is available at the 15th pick.  The Eagles would only make the trade if he was actually available at 15, so we don’t have to worry about that second probability.  Just note that it’s basically a flag that says: even if the Eagles and Steelers want to make that trade, there’s a 50/50 chance it won’t happen.

So, if the Eagles want Beckham, it looks like they really do need to move up.  How about the other guys in that tier?

There is a 51% chance that Marqise Lee will be available at 22.

There is a 21% chance that Brandon Cooks will be there at 22.

There is a 92% chance that Kelvin Benjamin is there at 22.

Now, these aren’t completely independent probabilities, so what I’m about to do isn’t 100% “correct”, but it’s not unreasonable either.  Combining those probabilities leaves us with a:

.49 * .79 * .08 = .03 or 3% chance that none are available.

So, the Eagles can give up their 2nd round pick for a 100% chance of Beckham, or they can keep their pick and have a 97% chance at one of the other three guys in that same tier.

Now can you see why trading up is such a terrible value?  We’ve already covered the margin of error issues.  Regardless of which players are in the same tier, conceptually they are all worth the same “value”.  So if the Eagles tiers looked like mine, they’d essentially be trading a 2nd round pick for a 3% increase in the odds of getting a WR from their desired tier.

That’s also known as a catastrophically bad use of resources.

Now let’s look at it a little differently.  Let’s say the Eagles do have Beckham rated significantly higher than the other three guys in that tier.  The operative question then becomes: how much higher?

This is important because we have to account for the opportunity cost of the 2nd round pick (which is large).  That brings me to the concept of saturation drafting.  In short, there’s no rule against using multiple picks in one draft on the same position.  For example, let’s say the Eagles have decided they NEED a star WR out of this draft.  They can:

A) Do Peter Kings trade, after which their odds of gaining a star WR will be whatever the odds of Beckham becoming one are.


B) They can NOT trade their 2nd round pick, and use it on ANOTHER WR!

To examine this possibility we need to know who will be available in round 2.  Let’s move to the next tier on our list.  This one:

Screen Shot 2014-05-06 at 10.47.24 AM

What are the odds those players are available at the 54th pick?

Davonte Adams – 38%

Jarvis Landry – 62%

Cody Latimer – 8%

Jordan Matthews – 37%

Once again, they’re all in the same tier, and the individual margin of error means they each of roughly the same expected value.  Given the odds above, that means there is a:

.62 * .38 * .92 * .63 = .136  that none of those guys are available at 54.  Flipped around, that means there is a 86% chance one of those players will be available at 54.

Going back to our two options, that means the Eagles, spending the same amount of draft resources, can have:

100% chance of Beckham


97% chance at a player like Marqise Lee AND an 86% chance at a player like Jordan Matthews.

Now, if you NEEDED a star WR, would you choose option A or B?  That ignores a lot of other options (for example, you could trade up in round 2 to give yourself a 100% chance of a 3rd tier WR), but it lays out the conceptual problems with trading up for a WR in round 1.

That’s a very long way of saying what I’ve said before:  If you are going to trade up for anyone, ESPECIALLY a WR, you need to be extremely confident he’s much better than the next guy.  Realistically, I just don’t see how the Eagles could possibly be that confident.

Therefore, trading up for a WR is a very poor strategic decision.  Remember, you’re not picking players, you’re picking lottery tickets.  Each one carries a different likelihood of “hitting”, but they all have risk of busting.  All you’re trying to do is maximize your odds.

The Eagles rankings undoubtedly look different from the tiers I’ve used above, but it really doesn’t matter what names you put in which tier.  Unless the Eagles think the gap between Beckham (or whomever) is EXTREMELY LARGE, trading up doesn’t make any sense.

Lastly, I’ll leave you with some spider graphs (from mockdraftable.com):Screen Shot 2014-05-06 at 11.01.38 AMScreen Shot 2014-05-06 at 11.02.14 AM

Screen Shot 2014-05-06 at 11.03.16 AMScreen Shot 2014-05-06 at 11.03.55 AM

Screen Shot 2014-05-06 at 11.05.11 AM



More on DeSean

I told you there was more…today I’ll try to knock a few more things off the list.  For this post, I want to focus entirely on the 2013 season.  While it’s not a large sample, it does represent the “DeSean under Chip” timeframe.  Seeing as how Chip Kelly remains the coach, it seems as though this might be pretty relevant information with which to evaluate DeSean’s importance to the team for next season.

So just how good was DeSean’s 2013 season?

Yesterday I covered Approximate Value and Receiving Yards.  Instead of repeating the analysis, I’ll just say that DeSean had a 2013 AV of 11 and 1332 receiving yards.  That puts him in a tie for 10th in Approximate Value and 9th in yards.

That’s really good, but hardly spectacular.

BUT…(you knew that was coming), there’s more to consider.   Beyond the production, there’s the qualitative (for now) value that DeSean adds to the offense.  His unique talent opens up the offense for the rest of the team, with the benefit redounding to other members of the offense rather than appearing in his stat line.  Let’s go beyond the generic “his speed stretches the defense” and try to illustrate, with stats, what makes him unique.

Below is a chart illustrating the Catch Rate for a sample of WRs along with their Deep Pass Attempt %.  These numbers are from AdvancedNFLStats.com.  For the sample, I included every WR who had 40 or more receptions last season.

Screen Shot 2014-03-26 at 6.16.05 PM

I added the trend line to illustrate the correlation (value is -.25).  That’s fairly intuitive.  Longer passes are harder to complete, so the catch rate for “deep threats” should be relatively lower.

Now let’s play a game.  Go to the chart above.  Using just the information displayed, which WR would you want on your team?

I think the answer is pretty obvious, but it might depend on your personal offensive philosophy.  However, regardless of your philosophy, it’s clear there are only a couple of logically defensible choices.  Now let me try to guess which one you chose.

Screen Shot 2014-03-26 at 6.17.43 PM

I thought so.  Now for the fun part.  As you might have guessed, the player I highlighted above is DeSean Jackson.  He’s a fairly significant outlier.  In other words, given the frequency with which his targets were “deep”, we’d expect his catch rate to be much lower.

In fact, just two other qualifying players had Deep Pass Attempt rates above 40%, Torrey Smith and Reuben Randle.  Smith’s catch rate was 47.4%.  Randle’s was 52.6%.  DeSean’s was…65.1%.  Clearly, one of these guys is not like the others.

Now let’s sort by Catch Rate.  Here are the top 20 from last season (remember we’re only including WRs with 40+ receptions.)

Screen Shot 2014-03-26 at 6.23.47 PM

Notice anything interesting?  Look closely at the Deep Pass Attempt rate column.  Among the top 20, DeSean has, BY FAR, the highest percentage of deep pass attempts.  In fact, the next closest player above is Doug Baldwin at 32.9%.  That’s a huge difference.   Now let’s flip it slightly and sort by Deep Pass rate.

Screen Shot 2014-03-26 at 6.33.08 PM

So now we have all of the major “deep threats” represented.  Look at the Catch Rate.  Again, near the top of the list, DeSean blows everyone else away.  Also note that Riley Cooper now makes the list, with the 6th highest Deep Pass rate.  However, his catch rate is “just” 56.6%.  I say put “just” in quotations because that’s still a very good catch rate, as you can see from the chart.   But it’s nowhere near DeSean’s 65.1%.

So, last season DeSean had a Catch Rate similar to the best possession WRs in the game, just behind Wes Welker actually.  Of course, he did so while operating as a true “deep threat”.  That’s a unique, and extremely valuable combination.

You see, last year the Eagles were able to attack downfield with DeSean without taking the normal reduction in Catch Rate.  Obviously, that makes the offense extremely efficient and dangerous.  More to the point, not a single other WR in the game put up numbers anywhere close to DeSean, at least not while catching more than 40 passes.

Now, the caveats.  It was just one season, so it’s certainly possible that DeSean’s Catch Rate and Deep Pass Attempts combination is unsustainable.  We’ll just have to wait and see on this one.  Additionally, you could credit a lot of DeSean’s stats to the genius of Nick Foles.  That too is possible.  Of course, that’s a LOT of credit to give to a 2nd year QB with mediocre arm strength.  As always, the “truth” is probably a combination of several factors.  One of them, though, is DEFINITELY DeSean’s skill.

Is there more?  Well yes, lots more, but let’s keep things relatively short.  It turns out AdvancedNFLStats.com also tracks Expected Points Added.  I’ve used the Expected Points concept in great detail before, so I’m not going to do a full explanation here.  Anyway, here are the WRs who registered the highest EPA last season:

Screen Shot 2014-03-26 at 6.54.57 PM

Yep, that’s DeSean, ranked ahead of guys like Josh Gordon, Eric Decker, and Calvin Johnson.  On a per play basis, DeSean also ranked 3rd, behind only Anquan Boldin and Doug Baldwin.

Like I said yesterday, DeSean Jackson is a GREAT player.  We can argue about what he’s “worth” contract-wise.  We can argue about whether he fits the “culture” that the team is trying to develop.  We can argue about how he’ll perform as he gets older.  When it comes to production to date, though, there is no argument.  Since entering the league, Jackson has been among the most productive receivers in the game.  Last season, with Chip Kelly calling the shots and Nick Foles at QB, Jackson became a tremendously unique and valuable weapon.  Don’t lose sight of that.

Scheme Fit

My final comment is a short one.  There was some talk about DeSean not “fitting” the scheme that Chip Kelly is trying to run.  Without engaging completely, because I think that grants too much credit to what is a ridiculous argument, let me just say this:

1) Chip Kelly is a great, creative, perhaps genius-level offensive mind.

2) DeSean Jackson, as shown above, is a unique talent and the premier deep threat in the league.

I’ve pretty much proven the second statement.  If you’re going to make the “scheme” argument, it stands to reason then, that the first statement above cannot be true.  Put simply, if Chip Kelly can’t find a way to use DeSean Jackson in his scheme, then he’s not the offensive mastermind everyone believes he is.  Of course, he did find a way to use Jackson productively last season, so….I told you that argument was ridiculous.

 Update: I forgot to mention that DeSean ranked 11th by Win Probability Added (also from AdvancedNFLStats.com).  That’s good, but given the EPA numbers I expected him to be higher.  I haven’t quite figured out a reason for the discrepancy, hence why I didn’t talk about it.  However, I don’t want it to seem like I’m hiding “unfavorable” stats, so there you go.

Contextualizing Mr. Jackson

Another day, another confusing report about DeSean Jackson.  Until something actually happens (like a trade or release), I’m not going to spend too much time discussing it.  There’s just not much value in speculating when we’re clearly missing a lot of information.  However, I have noticed something worrisome over the past week.  Namely, people seem to be underrating DeSean Jackson.

Make no mistake about it, DeSean Jackson is a great WR and he cannot be easily replaced.  I find it remarkable that I feel compelled to write a whole post about this, but here we are.

D-Jax entered the league in 2008.  He’s played 6 mostly healthy season with the Eagles, with 87 games played.

Take a moment and make a list of the 10 best WRs over the last 6 seasons.  Seriously, jot it down.  Is DeSean Jackson on there?  Probably not, and I’m guessing many of you didn’t seriously consider putting him on there.  Was that a mistake?

Following is a list of the top WRs over the last 6 seasons, as measured by Approximate Value.  AV is certainly not a perfect measure, and I don’t mean to suggest it is.  However, it’s a very good one, especially for offensive skill players.  Anyway, here’s the list:

Screen Shot 2014-03-25 at 6.11.22 PM

Now go back to your list of the 10 best WRs.  If you actually made it, I’m guessing there’s a LOT of overlap with the top ten WRs by AV.  Of course, the point of this exercise is to highlight where DeSean Jackson is (and is not).  By AV, he ranks 8th (tied), ahead of Larry Fitzgerald, Anquan Boldin, and Greg Jennings.  He ranks just behind Steve Smith.

The point is NOT that Jackson is a better WR than those players, that’s a different discussion.  The point is that he has been among the best WRs in the game since entering the league.  Note that Jackson gains AV points for his Return work as well, inflating his WR ranking.  Of course, it doesn’t make much sense to hold that against him either.

Let’s stick with AV for a moment, but take a slightly different look.  I’m always bothered by the “since he’s been in the league” ranking, because it’s a very arbitrary measure, typically skewed in favor of the subject player.  So, let’s change our filter.  Instead of looking at all WRs from 2008-2013, let’s just look at the first 6 years of each individual player’s career, going back to 1990.  Here’s the top 20:

Screen Shot 2014-03-25 at 6.18.46 PM

Spend a minute examining that list, and note that it underrates Michael Irvin by virtue of the 1990 cutoff date.  Regardless, notice anything?  Nearly half of them are sure-fire Hall of Famers.  Most of the other half at least deserves consideration.  Now you’re starting to see why I think giving Jackson up for little return is insane.  In terms of pure production, DeSean Jackson is a GREAT receiver. Full stop.

BTW, here’s the same filter, but sorting by Receiving Yards instead of AV.  Again, we’re looking at the most Receiving Yards over the first 6 seasons of a player’s career, for all seasons since 1990.

Screen Shot 2014-03-25 at 6.25.38 PM

DeSean has more yards in his first 6 seasons than Roddy White, Isaac Bruce, and Antonio Freeman.  Terrell Owens had just 53 more yards in his first 6 seasons, and actually averaged fewer yards per game.

Like I said…GREAT, and apparently under-appreciated.

Now let me attack the “replaceable” nonsense.  There seems to be an argument out there that basically says: “This is an extremely strong draft for WRs, we can just draft his replacement then.”  This is a fairly ridiculous argument.

1) Draft prospects are not guarantees!  Remember the post I did on Hacking the Draft?  Let’s take another look at the chart from that study.

See the odds of landing a “starting” WR with a 1st round pick?  Just 65%.  It’s possible that this year’s draft has a higher probability of success.  I’m willing to accept that assertion, but how much higher?  Certainly, given what we KNOW about the draft, not all of the WRs drafted in the 1st round this year will pan out.  Pulling from the studies I did on Skill vs. Luck in the draft, we ALSO know that there’s really no reason to believe the Eagles are more likely than any other team to be able to identify the WRs who WILL pan out.

So if you were planning on replacing D-Jax via the draft, think again.  It’s very possible, but it’s far from guaranteed, and we haven’t even accounted for the opportunity cost of a 1st round draft pick.

Let’s say you DO “hit” on a WR if the first round.  Surely then you’d have adequately replaced D-Jax, right?  Not so fast.

Here is a list of the best rookie WR seasons, by AV, since the 2005 season.  Note that, last season, DeSean registered an AV of 11.

Screen Shot 2014-03-25 at 6.37.11 PM

Since 2005, just ONE rookie WR (Harvin), has registered an AV as high as DeSean did last season.  Moreover, look at the column labeled “Draft”.  12 of the top 20 seasons by rookie WRs were from players NOT drafted in the first round!  Like I said, finding a great WR in the draft is NOT EASY, and certainly not just a matter of taking the best available guy in the first round.

In fact, since 1990, just THREE rookie WRs have registered AV’s above 11.  Randy Moss, Terry Glenn, Joey Galloway.  That’s the list.  23 years, 3 players.  Again, you’re probably not replacing DeSean’s production with a rookie from this year’s draft, at least not next season.

For those of you who don’t trust AV, I’ve also looked at receiving yards.  Here is the list of the best rookie WR seasons, since 1990, sorted by receiving yards.  For reference, DeSean had 1332 receiving yards last year.

Screen Shot 2014-03-25 at 6.43.43 PM

This is going to be tough, but go ahead and count how many rookie WRs had more than 1332 receiving yards….finished yet?  Thought so, counting to ONE doesn’t take that long. Note also that just 8 rookies have exceeded 1000 yards receiving. Remember, we’re looking at every WR in the league since 1990.  That’s a long time.

There’s a lot more to say about DeSean, but I’m going to stop there.  The main takeaways from today’s post are:

1) DeSean Jackson is a GREAT player.

2) The Eagles will NOT be able to replace his production next season, even if they “hit” on a WR in the first round.

Roster Building with Seattle: Skill, luck, or something else….

So Seattle just stomped Denver in the Super Bowl.  Game sucked, commercials sucked, RHCP played air-guitar/bass.  At least Bruno Mars came through.  I didn’t really see anything particularly interesting to discuss from the actual game, other than to note that Pete Carroll made some very poor strategic decisions early on (the 4th down calls).  Of course, if he thought his team was much better than Denver’s (and it appears that was the case), then taking the points may actually have been the right call.  Remember, favorites want low variance.  Refer back to here.

I did, however, see something interesting when looking at Seattle’s roster composition.  Specifically, the great defense the team has put together is composed of players who were drafted with relatively low picks.  Here’s are the starters, plus Cliff Avril:

Screen Shot 2014-02-05 at 9.38.10 AM

Two 1st round picks, neither from the top 10, and one 2nd round pick.  What you’re seeing there is an incredibly efficient use of draft resources.  The vaunted secondary features two 5th round picks, a 6th round pick, and Earl Thomas.

If this isn’t surprising you, remember that elite players come mainly from the 1st round of the draft, and almost entirely from the first two rounds.  See this chart from the archives:

So…it looks like what the Seahawks managed to do (build a historically good defense with low-round picks) should be extremely difficult, if not impossible. Let’s take a look.

First, how good were the Seahawks this year?

I’m going to use a shortcut here, and rely on just points allowed adjusted for league average.  This measure isn’t perfect because it doesn’t account for the contributions from the offense and special teams, but it also relies on readily available data, and more importantly, doesn’t offer a lot of false positives.  Since 2000, here are the best teams by this measure, with Seattle added to the mix:

Screen Shot 2014-02-05 at 9.15.55 AM

The teams highlighted in yellow won the Super Bowl.  I should also note that this year’s Carolina Panthers just missed the list.  They allowed 35.6% fewer points than average.

Anyway, as you can see, Seattle was historically good this year at preventing points.  Now, how does their roster compare?  Well I looked at the starting rosters, according to Pro-Football-Reference.com, of the 5 teams ranked ahead of them (haven’t had time yet to look the rest).  Here’s what I found:

Screen Shot 2014-02-05 at 9.53.22 AM

Screen Shot 2014-02-05 at 9.53.47 AM

Most of those teams seem to line up with our expectations; the best defenses ever feature a lot of 1st round picks.  I realize that not all of these players were drafted by the teams they ultimately played for above.  That’s a secondary matter, though, and one we can explore separately.  For now, I’m just interested in where the talent originally came from. It’s not enough just to count up 1st rounders.  We want a complete comparison that accounts for the differences in the value of each pick.  To do this, I used the Draft Pick Value Chart.  This is the chart teams either use or used to use as a guideline for weighing trades.  Here are each of the previously mentioned teams, with the starters listed along with their Draft Pick Value.  I’ve summed each at the bottom.  Click to expand.

Screen Shot 2014-02-05 at 10.05.08 AM

Here is the condensed version:

Screen Shot 2014-02-05 at 10.08.44 AM

So yes, the Seahawks appear to have done something remarkable, though perhaps not as remarkable as I expected.  Additionally, nearly all of Seattle’s contributors on defense were, in fact, drafted by the Seahawks, eliminating the possibility that they simply waited for late-round talent to prove itself in the league and then signed it.

The upshot:

There are a few possibilities here:

– The Seahawks are simply better at drafting then everyone else.  If you read the work I did on Skill vs. Luck in the draft, you know that’s almost definitely NOT the case.

– The Seahawks got lucky, and managed to string together several unlikely outcomes (very good players in the late round).  This one’s possible, and perhaps the most likely scenario.

– The Seahawks coaching staff is very good at turning its players into valuable contributors.  Also possible, but the level of results tells me to be skeptical here.

– Something else is going on….

We’ll need to explore these options in more depth to get a good sense of what’s actually happening.  For now, just know that what the Seahawks did, as far as roster-building goes, was EXTREMELY unlikely.  If we can figure out HOW they did it, it will tell us a lot about the kinds of strategies the Eagles should employ (even if it turns out Seattle did just get really lucky).

The Persistence of Sack Rate

Before we get to today’s topic, I need to clear something up about the last post.  It seems as though many readers came away with the wrong message. Nobody here (EaglesRewind.com) falls into that camp (at least nobody who commented), but Philly.com pulled the post from BGN and ran it on the front page of the site….

Hilarity ensured, at least if you read the comments.

So, let me make this clear:  I am a huge Nick Foles fan and the last post was not really bad news for him.  Yes, interception rate shows very little persistence from one year to the next, and that is, by itself, bad news for Foles (because he did so well this year and last).  However, I also mentioned that, (1) Foles rate was so good that even a relatively large amount of regression would still leave him with a very strong rate, and more importantly, (2) it looks as though some QBs are able to improve upon their interception rate over time.  That second part is a vital piece to this puzzle.  It means there is definitely skill involved in avoiding interceptions (hardly a surprise), but it also means that Foles may be able to actually improve his “true” ability level, which would obviously counteract some or all of the expected regression.

So, good news, not bad.

Now, for today, I decided to look at another aspect of Foles’ game thats drawn a lot of scrutiny: his sack rate.  

Foles has a career sack rate of 7.6%, and this season he was sacked on 8.1% of his drop-backs. (Pro-football-reference.com).

The first question, obviously, is:  Is 8.1% bad?

Well, it’s not good.  This season, it left Foles ranked 27th in the league, just behind Kellen Clemens.  There were a few notable QBs who did worse though, like  Cam Newton (8.3%), Colin Kaepernick (8.6%), and Russell Wilson (9.8%).

I know what you’re saying, those are all “running” QBs, and their high sack rate should be balanced against the positives they bring in the running/scrambling game.  (If you weren’t saying that, you should have been).  While I’ll attack the positive/negative balance another time (check Football Outsiders’ advanced metrics for a very surprising look), allow me to posit another potential explanation:  they’re all relatively young.

Here is the list of QBs who finished with a worse sack rate than Nick Foles this year, along with their ages (right column).

Screen Shot 2014-01-27 at 10.07.45 AM

That’s a lot of 23 and 24.  This makes complete sense, for one of two reasons.  Either:

1) Sack avoidance is a skill that can be improved over time


2) QBs who take a lot of sacks don’t remains starters for very long.

The most likely explanation is that its a bit of both (as it usually is).  If a QB takes sacks at a very high rate, it’s going to be very difficult to be a productive offensive player.  There are exceptions, though, like the QBs I named above (Foles, Wilson, Kap, etc…).  Also, Ben Reothlisberger has a career sack rate of 8.2%.  Clearly, a high sack rate is not catastrophic.

For more, look at this year’s leaders:

Screen Shot 2014-01-27 at 10.12.44 AM

Jason Campbell, Jay Cutler, Sam Bradford, Matt Schaub, Matt Cassel….There are a lot go great QBs on that list, but also a handful of mediocre-bad ones as well.  Once again, sack rate isn’t everything.

Now, to the title of this post, persistence.

In my last post, we found that interception rate persists at a relatively weak level (at least within our admittedly limited sample).  That means one year’s rate has little informational value regarding the following season.

Let’s do the same thing with Sack Rate.  Simply put, does getting sacked at a high rate one season (like Foles this year) mean you’re relatively more likely to be sacked at a high rate the following season.

The answer, overwhelmingly, is YES.

Here is the chart:

Screen Shot 2014-01-27 at 10.18.09 AM

Again, the sample only consists of 100+ attempt seasons from: Eli Manning, Peyton Manning, Matt Schaub, Mike Vick, Matt Hasselbeck, Drew Brees, Tom Brady, Philip Rivers, Matt Stafford, Carsoln Palmer, Tony Romo, and Aaron Rodgers.  All told, that’s 112 qualifying seasons.  The same caveats I discussed last time apply here as well.

From the above data, the correlation value is 0.57, which is VERY strong compared to most statistics in football.

Clearly, a player’s style and skill have a large effect on their sack rate.  More importantly, Nick Foles is likely to be sacked at a high rate next year.  By itself, that’s bad news, but not as terrible as it might seem.

First off, the youth factor.  Here is a chart showing the average sack rate of the entire sample by qualifying season.  Note that these are NOT weighted numbers, so Hasselbeck’s 10.6% rookie rate counts the same as Aaron Rodgers’ 6%, regardless of the number of attempts.

Screen Shot 2014-01-27 at 12.23.39 PM


There is a clear and significant drop-off after the first season.  Of course, this year was Nick Foles’ SECOND season, but he did start just 6 games his rookie season, so I don’t think it’s unfair to look at this year as something like a continuation of his first season (though we can’t ignore the fact that a full offseason SHOULD play a big role).  There are definitely a lot of other factors in play here, for example maybe teams tend to improve the offensive line in the first offseason after anointing a new QB.  So nothing here is definite.

Still, from the sample I looked at, it’s certainly possible, and perhaps likely, that Nick Foles will see a decline in his sack rate next year.  However, the high correlation value suggests that we shouldn’t expect a significant difference.

I’m going to leave it there for now because I already feel like I’m rambling a bit, but let me give you one last chart and correlation value to think about.

How do interceptions and sacks relate?  The “narrative” frequently says that it’s “better” to take sacks than to throw interceptions.  Alone, that’s almost always the case.  But, it seems to suggest that one can and does take sacks INSTEAD of throwing interceptions.  For example, everyone that says Nick Foles doesn’t throw interceptions BECAUSE he’d rather take a sack is making a logical jump and assuming that the two outcomes are related.  I need to do A LOT more research on this, but preliminarily, here’s a chart showing the correlation between interception rate and sack rate, within our sample:

Screen Shot 2014-01-27 at 12.33.41 PM


A jumbled mess.  The correlation value is just 0.123 (roughly the same strength as the persistence of Int rate).  Remember, if the “Sacks INSTEAD of INTs” narrative holds, we should see a negative correlation (so more sacks equals fewer interceptions).  That’s not what we have.  As explained, I need more data and a deeper dig before coming up with any conclusion I can feel comfortable about.  In all likelihood, we need to groups QBs by “type” and evaluate them within discrete groups according to their attributes.

One more thing about Foles

Not a full post today, just a point I had wanted to include yesterday but forgot about.

The lack of excitement/”believers” regarding Foles may be a direct result of his “elite” skill.  as I said yesterday, it’s possible that interception-avoidance is Foles’ “plus” trait.  If that’s true, he’s never going to command a huge following.

Think about it.  The best plays Foles makes are, by definition, the ones you DON’T see, simply because they don’t happen.  As a fan (or anybody), it’s impossible to identify the plays in which a replacement-level QB would have thrown an interception but Foles does not.  Sure, we can talk about it in the abstract, for instance when he takes a sack and we say “it’s better than an INT”.  However, that’s obviously never going to be featured in a highlight reel.

In particular, Foles might be hurt by the way the game is analyzed today.  With the prevalence of All-22 breakdowns, countless people are going through game-tape and identifying everything that happened.  Once again, interceptions NOT thrown is never going to show up on those.  Instead, you’ll get a handful of wide open receivers that Foles missed.  The take-away, naturally, will be about what an “elite” QB WOULD have done, given the same openings.  However, that type of analysis doesn’t account for Foles’ “elite” skill.  Yes, maybe another QB would have made a few more plays.  However, he also may have then turned the ball over, completely negating the additional positive plays.

I’m going to try to dig a bit deeper into both interception rate and sack rate.  Hopefully, I’ll find evidence one way or another indicating the degree of skill involved in each.  Until then though, keep in mind:

IF Foles’ “elite” skill is his ability to NOT throw interceptions, he will never be fully appreciated.  Not only is it not a real “measurable” skill, but it doesn’t show up on replay.  While other QBs may be able to make a lot of positive plays that Foles can’t make, Foles may AVOID a lot of negative plays that those other QBs don’t.

That’s not as easily identifiable and it doesn’t lend itself to highlight reels, but in theory, there’s no reason it can’t be just as significant a skill.