Prospect Rankings via Positional Value Multiplier

Posted on March 7, 2013 by Brent

Today, with the help of a collaborator, I’ll give you prospect rankings for the NFL Draft that you won’t find anywhere else. As I’ve explained before, I am not a scout and have not watched film on every top prospect in this year’s draft. However, I believe what I’ll show you today is more useful than any individual scout’s ranking.

First things first, big thanks to George Laevsky (JD from Georgetown) for the help. He came up with the idea and name for the Positional Value Multiplier and worked with me on compiling/computing the necessary data.

To keep this clean, I’ll explain it in 3 sections. First I’ll tell you what we did, then I’ll tell you how we did it, then I’ll show you the results. That way, if you want to skip the middle section you can.

What We Did:

The overall aim of this project was to apply a positional value modifier to the consensus prospect rankings, with the hopes of generating a more accurate system of ranking value. We compiled a composite prospect rating for each player (through the first couple rounds) and then adjusted for positional importance according to last seasons’ league-wide positional salary distribution.

Before we go into the How details, here is the consensus prospect ranking using ratings from Scouts Inc (ESPN), the National Football Post, and NFL.com. Note: NFP uses a different grading scale, so those scores were adjusted to give us an apples-to-apples rating.

That graphic alone is pretty interesting, particularly when the ratings diverge (see Ryan Nassib at the bottom), but we’ll look at that some other time.

For today’s post, we have to adjust.

How We Did It:

I mentioned last week that no BPA ranking is complete without an adjustment for relative positional value. For example (an extreme one), if a QB and K both carry a 95 rating, you’d obviously choose the QB first. The question is, how do we measure relative importance by position?

While there is no bullet-proof method of doing so, the salary distribution in the NFL is as good a place as any to divine information from. In theory, since the NFL has a salary cap, the distribution of limited funds between positions will give us an idea of how the league, on average, values different positions in relation to each other.

We pulled salary cap information from this awesome graphic featured in the Guardian at the end of January. It’s not perfect (reflects cap hits from last season and misses some IR guys), but in general I believe it’s as good a breakdown as any for our purposes today. After adjusting for the number of players by position, we calculated a Positional Value Multiplier (“PVM”) for each major position (FB, K, P not included). We then applied that multiplier to the above consensus rankings.

Here are the multiplier values we arrived at, in order from largest to smallest:

For the most part these make a lot of sense, based on what’s “common knowledge”. QBs are, by far, the most important position. However, the relative rankings of WRs and RBs certainly surprised me, though due to the noise in the data, it’s best not to get hung up on the minute differences in values above. Instead, we can see there are some clear “tiers” (I feel like I am using that term a lot).

Tier 1 – QBs

Tier 2 – WR, CB, DE, RB, DT

Tier 3 – OT, LB, TE, S

Tier 4 – C, G

The only thing in those rankings that immediately draws my attention is the OT position in the 3rd tier. But that data is what it is, we can debate the reasons later.

Now that we have the PVM values, we can apply it to the prospect rankings.

The Results:

Some very interesting movement. The right-most column shows the effects of the positional modifier. The AG Rank column is the pre-adjustment consensus ranking.

Notes:

– Dee Milliner jumps two places to become the top overall prospect.

– Chance Warmack, though he drops 3 spots, remains a top 5 prospect, damaging my belief that a G in the top 15 picks is a very poor decision.

– QBs, as expected, benefit the most. Geno Smith jumps 11 spots to become a top ten prospect, while Nassib and Barkley move into the middle of the first round.

– The biggest jump overall comes from Tyler Wilson (20 spots), who moves from the middle of the second round to the end of the first.

– Zach Ertz (TE) and Jonathan Cyprien (S) are hurt the most, falling out of the first round, and therefore off the chart above.

One last thing: I want to be perfectly clear about the value of this analysis. The idea here is that BPA is an overly simplistic and flawed method of drafting by its current definition. For example, while Geno Smith (19th consensus) may be a worse prospect than Kenny Vaccaro (9th consensus), with lower odds of success, the potential payoff is so much greater for Smith that he becomes a better choice (at least as shown here). Hitting on a QB offers a MUCH greater reward than hitting on a S (or really any other position), so it makes complete sense that QBs are perennially “over-drafted”.

In essence, what we are showing here is that they are not, in fact, “over-drafted”. Yes, they might have greater odds of failure, but that does not make them bad picks. Remember, you have to look at both Risk AND Reward, balancing the two. The above rankings is an effort to do that in a method as simple and transparent as possible.

Over the past few months, I’ve tried to advance the idea that the “consensus” forecast should carry a large degree of inertia within NFL front offices. Imagine the above rankings as equivalent to a total market stock index. For anyone going against the total market index, they must believe VERY STRONGLY that they have better information or better analysis than the rest of the market. It should function in much the same way in the NFL (and all professional sports leagues). The idea is NOT that teams should blindly follow the “market”, just that they should hold their own evaluations up to very high scrutiny before acting on them, especially when they largely conflict with available data.

I’ll be examining this in a lot more detail, which may or may not lead to more posts on the subject. In any case, this should give everyone something to think about come draft day.

For what it’s worth, my subjective pick for the Eagles would still be Lotulelei/Joeckel. However, unless I adjust the PVM formula (or if the consensus ratings change), it looks like Dee Millner is now, objectively, in the lead.

UPDATED: Also, below is the same analysis for the rest of the players we looked at:

How often do “Can’t Miss” prospects miss?

Posted on March 5, 2013 by Brent

I previously stated that “no prospect is risk-free”. Today we’ll look into just what that means.

The Data:

I went back through every draft from 2005-2011 and compiled the Scouts Inc. grades on each player taken in the first round. According to the Scouts Inc. grading scale, any prospect with a 90+ grade projects as a potential “elite player”. Recognizing that 10 points on a 100 point scale is a pretty big range, I will only look at prospects rated 95 or better. Theoretically, these players should be the absolute best of the “elite” prospects and should carry a very low miss rate. We’ll look at each rating and assign players to 1 of 3 categories: Hit (Those who can be considered elite players), Bust (completely non-productive players), and everyone else. Forewarning, I’m going to err on the positive side but feel free to assign your own if you disagree. At the end, we’ll total up the categories and see what happens.

Note: These types of breakdowns typically devolve into parsing Hits/Busts, as everyone will have their own opinions on which players belong in which categories. This is not my goal. My hope is that while there will be disagreements over a few of the ratings, the overall numbers will provide useful insight. Remember, this is not an individual player analysis, it’s an attempt to draw insight into the broader “elite prospect” pool. One or two ratings changes won’t matter that much.

Also, we are much more interested in the “Busts”, so moving players from “Hits” to “Everyone Else” or vice-versa doesn’t really effect what we are looking at.

The 99s:

We’ll begin with the absolute best, those players receiving a 99 pre-draft grade. Here they are:

Hits: Aaron Rodgers, Calvin Johnson, Andrew Luck (a bit early but remember we’re being generous).

Busts: None

The 98s:

Hits: Ferguson, Thomas, Peterson, Jake Long (on the edge due to last year), Matt Ryan, Gerald McCoy

Busts: Adam Jones, JaMarcus Russell, Matt Leinart

The 97s:

Hits: Suh, Von Miller, A.J. Green, Patrick Peterson, RG3, Marcell Dareus

Busts: Mike Williams, Brady Quinn, Jason Smith, Curry

The 96s:

Hits: Ngata, Keuckly

Busts: Sims, Quinn, Rivers, Gabbert, Maybin

The 95s:

Hits: Merriman (was elite while on roids, so i guess that counts), Willis, Revis, Earl Thomas (ugh)

Busts: Travis Johnson, David Pollack, Alex Barron, Jamaal Anderson, Amobi Okoye, Derrick Harvey

The Totals:

The 99s: Hits – 3/5, Busts – 0

The 98s: Hits – 6/14, Busts – 3/14

The 97s: Hits – 6/21. Busts – 3 4/23

The 96s: Hits – 2/21. Busts – 4 5/21

The 95s: Hits – 4/30, Busts – 6/30

In looking at those totals, it’s clear that there is actually a difference in success odds for each discrete rating, which really surprises me (though it’s hardly enough data to confirm). In general, there is a clear decrease in Hit odds and corresponding increase in Bust odds as you move from 99 to 95.

However, it also clearly shows what I was originally hoping to illustrate: There is no such thing as a risk-free prospect. I realize that none of the 99s rated as busts, but since there are only 5 of them, and given the Edwards and Bush careers, I think its safe to say we will see a 99 bust at some point.

In total there were 93 prospects included that rated 95 or better. Of these, I rated 16 18 as Busts, or 19.35%.

So examining the absolute best of the best prospects, almost 20% of them end up busting. I realize this is a very subjective breakdown, but even with a few changes either way, the overarching point remains, even the best prospects have a significant chance of complete failure.

BPA absent Positional Value adjustments

Posted on March 4, 2013 by Brent

Last post I attempted to explain, in no uncertain terms, why taking a Guard in the first round makes little sense (especially in the top half of the first round). However, I get the sense that many people remain unconvinced. Today, I’ll take a slightly different look at it to see if I can sway the remaining holdouts. I apologize for the repetition in subject-matter, but this is an EXTREMELY important topic, as it goes to the heart of Optimal Draft Strategy.

First, I want you to ask yourselves if you agree with the following statement:

GMs should, in general, take the Best Player Available (“BPA”) with every draft pick.

I’m guessing most readers here would back that strategy, and I certainly do. HOWEVER, it is not enough to just endorse that statement and use it as the basis for a draft strategy. First, you must definite exactly what BPA means.

Here is where I am seeing some confusion and where I believe there is a big disagreement. The people calling for taking Warmack high in the draft (forget about the Eagles for a moment, I’m speaking in broader terms), are using BPA as support. The problem is that, in their BPA definition, they do not seem to be adjusting for positional value, which is a MASSIVE mistake.

Rather than attempting to explain why with a rational argument (for that see my last post), I’ll try to illustrate it. Here are the top ten OTs and Gs from this past season, according to Pro Football Focus. They are listed in order, with their original draft pick included.

There are two big takeaways from this chart. First, look at the draft picks.

I know I’ve said this repeatedly, but if you want an “elite” OT, you have to take him in the first round. The same clearly does not hold for the Guard position. This provides a clearer illustration of the opportunity cost argument I made last week. There are really only two ways to get an elite OT in the NFL, draft him or sign him in FA. (I know we traded for Peters, but that’s a RARE exception). Additionally, signing an elite OT in FA requires a huge contract. Therefore, the BEST way to add a top OT to your team is to draft him in the first round. Choosing a G means you are forgoing that opportunity.

The second point I want to highlight from the chart above gets at positional value. Again, I want you to answer a question:

All other things being equal, would you rather trade Anthony Davis for Evan Mathis (or any other top Guard) or vice-versa?

I’m a big Mathis fan, but that’s a no-brainer. Clearly most of the league agrees as well, hence the salary differences between OTs and Gs.

Assuming you agree, that means the #10 OT is worth more than the #1 G.

How about Brandon Albert versus Evan Mathis?

This one probably creates a bit more disagreement, but my guess is, on the whole, people would take Albert, who ranked as the 25th best OT according to Pro Football Focus.

Regardless, clearly there is a discrepancy in value between OTs and Gs. Also, while I’ve focused on Gs and OTs, the same analysis can (and must be) done with all positions, and then incorporated into each team’s prospect rankings. To not account for this is a huge mistake and one that appears is made by a lot of the writers who support taking a G high in the draft under a flawed concept of BPA.

To make sure I’m being clear: This should be incorporated into the “tiered” rankings I advocated previously. So it is theoretically possible for a G to be the best choice with a top 15 pick, but it would require an amazing G prospect that can overcome the positional value difference to make it to the top 1 or 2 tiers, as well as every other player (from positions of greater value) within the same prospect tier being taken before said top 15 pick.

Hopefully this has convinced a few more of you, or at least provided a clearer explanation of what I was getting at last week. This isn’t exactly Eagles-specific, because I don’t think there’s any way they take Warmack at #4, but it applies to any team with a top 15 pick.

Lastly, I’ll leave you with an analogy. A frequent defense of G picks (and OL picks in general) in the first round has been that they are “safer”. In general, that is true, as our strategy chart showed. However, this is only one half of the equation. As anyone in finance knows, evaluation is always a question of risk vs. REWARD. To look at one side and ignore the other is a recipe for sub-optimal decision-making.

For example, given the choice between to raffles, and ignoring an external factors, which would you rather enter:

A) 60% chance of winning $100.

B) 75% chance of winning $75.

While the odds of success for raffle B are significantly higher, the correct answer (in a vacuum) is A, since the expected payout is greater. It’s impossible to apply such specific values to players, but it’s a useful exercise nonetheless. Here, imagine a DE as raffle A and an OL as raffle B. To take the OL just because he offers less risk (25% chance of miss versus 40%) would clearly be a poor decision.

Dispelling the Chance Warmack Nonsense

Posted on February 28, 2013 by Brent

I’ve been asked by a few people about my thoughts on Chance Warmack, so today I’ll attempt to provide a definitive explanation for why taking him with the 4th pick (or any high pick for that matter) is almost definitely a terrible idea (I’ll explain the one scenario that makes sense as well).

Although I have been clear about the Eagles not taking him, I haven’t fully explained, and judging by this article (Don Banks mock draft), there needs to be a reasoned analysis.

The first big point to consider is that Warmack is NOT risk-free. No draft pick is, regardless of what scouts believe. While some picks/positions are clearly LESS RISKY than others, every selection carries a chance (I really wish that wasn’t his name) that the player won’t pan out.

Let me provide a few quick examples. Here are some pre-draft scouting reports, followed by the corresponding player:

One of the more versatile offensive tackles to enter the draft in a long while, [He] is a terrific prospect that will offer immediate returns as well as long-term potential. Should quickly come in and start as a left tackle for a franchise, yet could also grow into a dominant strong-side blocker. Will only get better as he physically matures and improves his skills as a run blocker. SI.com

The Player? Robert Gallery, selected #2 overall.

“I can’t remember being in such awe of a quarterback in my decade of attending combines and pro days. [His] passing session was the most impressive of all the pro days I’ve been to. His footwork for such a big quarterback was surprising. He was nimble in his dropbacks, rolling out and throwing on the run. The ball just explodes out of his hands.” – Todd McShay

The subject? The immortal JaMarcus Russell.

Athleticism at the…position is unmatched by any player drafted in the last decade. Incredible vision…Surprising power and is not afraid to run people over. Absolutely chiseled physique…good character guy…Threat to score every time he touches the ball, and he should do that a lot in the NFL…Rarely does an athlete of this caliber enter the NFL. -draftclass.com

Player? Reggie Bush

Obviously there are a lot of players that we can look back and poke holes in, but the fact is each of the above players was considered very low risk at the time they were drafted. I’m not saying Warmack will be a bust, odds are he’s gonna be very good, but let’s just remember that scouting reports don’t always pan out.

The second and more significant reason for not taking Warmack (or any guard) is Opportunity Cost. When you use a draft pick, you are not just selecting a player, you are also deciding NOT to use that pick on a different prospect(s). You can’t evaluate each choice in a vacuum, you must look at what else is available, otherwise the whole idea of “value” gets thrown out the window.

Remember our rule of thumb regarding the draft: Elite players come from the first round, and mostly from the top half of the first round, REGARDLESS OF POSITION (with a few exceptions like K, P, FB, etc…)

Before we fully consider that, let’s also bring our draft strategy card into the picture:

Sorry for the small font, but there’s a lot of info in there. To refresh, the chart shows the odds of success by position and round for prospects taken in the draft from 1999-2011.

By combining our elite prospect guideline with this chart, it’s pretty easy to see why taking a guard is a bad idea in the top of the first round. If we gauge the relative impact by position, Guards come out near the bottom (which is why so few are drafted highly). The fact is, a dominant DE or T will have a greater effect on the game than a dominant G. Also, there are very few opportunities to add elite players at high-value positions, with the Draft being, by far, the best route for all teams (as a result of the contract value).

Therefore, choosing Warmack means you are forgoing the RARE opportunity to add an elite player at a more valuable position. If it was easy to find top OTs or DEs or DBs, then taking a G wouldn’t be such a big deal. Unfortunately, if you don’t use the draft for the high value positions, it becomes VERY difficult and almost ALWAYS expensive to fill them through other avenues (FA, Trades).

I should note at this point that this does depend on the draft board. I am a firm advocate of BPA strategy. However, there are enough positions of higher value than G that I’m extremely confident there are better options than Warmack at #4. Remember that we shouldn’t be ranking players individually, but rather slotting them by tiers to account for the large margin of error in player evaluations. So looking at the tier that includes Warmack (varies according to team board), there is a very high likelihood that, at #4, there is another player within that tier that plays a more impactful position.

Lastly, if we look at the chart, we can see that Guards are among the easiest to find late in the draft (highest probability). In the 6th round, for example, 21% of guards selected from 1999-2011 became starters (started for at least 5 seasons or half of their time in the league if they were drafted less than 5 seasons ago). Conversely, just 5% of DEs and 4% of DBs taken in the 6th became starters. While players selected that late are very unlikely to be “elite” players, you’d clearly take an Elite DE/average G combo over the reverse.

To sum up before I get to the one exception scenario, the goal in the 1st round of the draft for EVERY team needs to be MAXIMIZING IMPACT. Each year 15 teams (top 15 picks) get a reasonable shot at adding an “elite” player. Outside of that draft choice, the opportunities to add elite players are few and far between. Most often, the only other option is for teams to overpay in free agency, especially for premier positions. Therefore, teams cannot squander their best opportunities to add elite players on lower-impact positions, regardless of how strong the prospect is.

I mentioned at the top that there was one (extremely unlikely) scenario where it made sense to draft Warmack:

BOTH:

– Warmack would have to be in his own tier, and CLEARLY a far better prospect than anyone else on the board. From what I’ve seen, that is not the case.

– There would have to be NO interest in trading for the Eagles pick, eliminating that as an option. Even a trade involving relatively less compensation than normal would be enough to move, so I really do mean that a trade would have to be nearly impossible. Again, the odds of there being ZERO interest in anything approaching a reasonable offer for the #4 pick are very low.

OR – the subject Guard (not Warmack) would have to be so amazing as to completely redefine the Guard position. I’m not even sure what that would look like. Safe to say it’s not happening anytime soon.

So for the time being, ignore Don Banks and anyone else suggesting the Eagles should/will take Warmack.

Alex Smith Thoughts and ramifications for the Eagles

Posted on February 27, 2013 by Brent

I already liked the Chiefs to be a drastically better team next year, and now I like them even more. The reported trade has KC giving up this years 2nd round pick and a conditional 2014 pick. No word on the conditions, but it’s likely going to be as high as another 2nd round pick.

Why I like the trade:

– Andy Reid is very good at identifying QBs he can win with. That’s not the same as identifying good QBs, but from Reid’s perspective, that really doesn’t matter. Smith is a perfect fit for the offense Reid claims to run (west-coast). I say “claims” because there were times with the Eagles when he clearly strayed from the short-passes concept, relying on deep throws and quick strikes.

– Kansas City already had a top 5 rushing offense. Now they add an accurate, low-risk QB, and the #1 overall pick (Joeckel has to be odds on favorite now).

– Alex Smith went 19-5-1 as a starter over the last two seasons, and this past year completed 70% of his passes.

I could go on for a thousand words, but I’ll just go on record now as having the Chiefs as my pick for a surprise team next year. With that roster (better than record suggests) and Reid/Smith, no reason the team can’t contend for the playoffs just one year after a 2-win season.

How does it effect the Eagles?

– Nick Foles’ value obviously drops. KC was, by far, the most likely destination for Foles. That is the clear negative for the Eagles. Additionally, KC will almost certainly take Joeckel (or whoever the top OT is at draft-time), meaning the Eagles will miss out. However, it’s doubtful the top OT in the draft was going to fall to #4 regardless, so I don’t feel so bad about that.

Also, I’ve been pretty clear that I think Foles has a legitimate chance to be a good starting QB, so keeping him isn’t exactly the worst thing IMO.

– One less team looking for a QB in the draft. As I explained in the Vick-Insurance post, it would make a lot of sense if the Eagles had their eye on a particular QB in this year’s class. Nassib would seem to be a good bet, but at this point we have no idea who the team really likes. The Smith trade makes it more likely that the subject QB will be available at the Eagles pick in round 2.

– Several outlets are now reporting that San Fran is an obvious candidate to move up in the draft. As I’ve demonstrated before, elite players come almost entirely from the 1st round of the draft, and within the first round, they are predominantly found in the top 15. The toughest aspect of being perennial contender (like San Fran hopes to be) is the inability to continue adding elite players in the draft (because to don’t get top 15 picks). With San Fran’s ammo, they can rectify that this year.

I mention that because the Eagles, at #4, would seem to be an obvious trade partner. At #4, the 49ers could get a top DT (Floyd or Star, depending on the heart condition). If the Eagles don’t see any prospects that absolutely love at #4, the team should absolutely trade down, and the top pick in the 2nd round would be a pretty attractive trade chip. In fact, this would allow them to take a QB prospect at the right spot in the draft (value-wise), while allowing them to use the other 2nd round pick on the best available defensive player.

Pure speculation at this point, but the Chiefs, Jaguars, and Raiders all seem unlikely to trade down, given their respective needs. That would leave the Eagles with the top “gettable” pick for any team looking to move up for a top prospect.

Judging by the current prospect evaluations (and what we learned from our previous draft analyses), that trade outlook might be the best possible scenario for the Eagles.

Turns out Andy Reid may not be quite finished helping the Eagles out…

Round 2: Luck vs. Skill in the NFL Draft

Posted on February 26, 2013 by Brent

Round 1 looked at historical draft performance and persistence based purely on total production as defined by CarAv. Today we’ll make an adjustment to the model and rerun the analysis to see if anything changes.

For reference, Round 1 found absolutely no correlation in team performance from one year to the next.

The adjustment for round 2 is accounting for draft position. Using the default NFL Draft Chart, I assigned a draft points value to every player selected from 1999-2009. I then summed up the total draft points used by each team each year, and divided that number by the total CarAv to arrive at a Points per CarAv value, which I then used to rank teams each year.

This should sound very familiar, as it is similar to the method I used in our first draft performance ranking. However, while that analysis looked at best performance over the whole length of time, this is only concerned with performance each individual year. As such, we don’t have to adjust for the cumulative nature of CarAv, so the rankings are a bit different.

Before we start parsing the results, here is the new ranking:

This one definitely passes the eye test. At the top of the board are Indy, Baltimore, Green Bay, Pittsburgh, and the Giants. Notice anything in common?

Meanwhile, at the bottom of the board are Detroit, Oakland, Cleveland, and Arizona.

So before digging in, it appears to be a reasonable ranking according to what we know of each team’s performance.

The Caveat – Using this system accounts for teams that perennially pick at the end of the draft (or the beginning), that way a team like the Patriots doesn’t get penalized for not finding similar quality players as a team like Cleveland, which, on average, picks much higher in the draft. However, using the NFL Draft Value Chart does have a drawback. The chart is HEAVILY skewed towards the top of the first round. For instance, the #1 pick is worth 3,000 points while the last pick in the draft is worth just 2. The upshot is that using it to adjust for team draft position skews the results in favor of teams who do not have a lot of Draft Points (i.e. good teams that don’t pick high in the draft or teams that don’t have 1st round picks). This is an area that I will try to adjust for in a future analysis.

Luck vs. Skill – Remember that if the draft is mostly skill, there should be positive correlation between team performance one year and the next. How much correlation? That’s up for serious debate. Regardless, here is the chart followed by the correlation value:

Doesn’t look like much, so I added a trendline to make it clearer. The value is .167. Not huge, but significant, and certainly a different story from the last look, which showed no connection.

If we take a step back, this comes out where I imagined the luck vs. skill argument to be when I started the original analysis. I believe that there is obviously skill involved in the draft, but that luck is far more determinant of team success in this area of the NFL. In general, my theory is that GM skill is illustrated by moving around the draft to maximize value/odds, and not necessarily in selecting players. A lot more to do to support that, but I think this analysis is a step in the right direction.

Other notes from the analysis:

– It should come as no surprise that, according to this analysis, the Eagles 2002 draft remains its best. Overall, the team’s average ranking over the subject time period was 15th.

– The Eagles were one of the most schizophrenic teams when it comes to draft performance, with a more volatile ranking than all but 3 teams (Jets, Miami, and Denver). The Broncos were least consistent.

– By this measure, the Eagles worst draft during the Andy Reid Era was 2003, when the team spent 26.71 draft points per CarAv. For comparison, in 2002 (best draft), the team spent just 6.15 draft points per CarAv. For those who don’t remember, 2003 was the Jerome McDougle, L.J. Smith draft.

– Over the subject timeframe, the Eagles have spent 3.74 draft points LESS than the league average per CarAV, which ranks 9th best in the league. (Chart below)

– The Colts spent 8.70 LESS draft points per CarAV than league average, while the Lions spent 13.05 points MORE than league average…

– The Detroit Lions have been shockingly inept when it comes to the draft. The teams average ranking is 26th, but for a clearer illustration, here are the Lions 1st round picks from 1999-2009:

Remember the post on how important it is to hit in the first round? Needless to say, the Lions blew it…repeatedly. To be fair, the team’s most recent #1s look OK (Suh, Fairley)…if you forget about Jahvid Best.

– The Patriots have fallen off dramatically. From 1999-2003 the team’s average ranking was 10.6. From 2004-2009 its average ranking jumped to 19.

– While the above analysis looked at average ranking, here is a chart showing performance over the whole time period, as defined by average draft points per CarAV minus league average. (So average team performance versus league average). Negative numbers = better performance.

There’s a lot more I can do now that I’ve put the information together (which took much longer than I expected). So expect to see a few more points from this analysis.

Overall, the evidence still stands heavily in luck’s corner in its fight with skill for the soul of the NFL Draft, though it’s fair to say skill has landed a couple jabs.

Scattered thoughts

Posted on February 25, 2013 by Brent

– Historical Draft Performance

The last post was a pretty broad look at draft performance. I’m currently making a few adjustments to the analysis that will hopefully provide a more accurate ranking system (though the existing version passes the eye test). If we get an accurate system, it’s a small step to mine it for performance persistence (skill). Adjusting for draft position is the big addition, but I’m also looking at relative magnitude (team share of all production available in draft). If you have any other ideas for making a more accurate ranking system let me know in the comments and I’ll see if I can work it in.

– Sean Smith Interest

This is exactly the type of fake news you should avoid. The Eagles are interested in the top CB available at the right price? Shocking. Saying you’re interested in a top FA but don’t want to get in a bidding war is ridiculous. That’s how free agency works (for the top players). However, the Eagles do need to add a CB, preferably one that can start, and adding a mid-level guy in FA would make me feel a lot better about the roster for next year. If they really do like Smith, they’ve got the cap space to make the move and win a bidding war.

– Nnamdi

Anyone who was against tagging DRC should be completely aghast at the potential for bringing Nnamdi back with a restructured deal. From what I saw, most people don’t like DRC because of his inconsistent effort. I challenge you to watch Nnamdi’s tape from this year and tell me you think he was playing hard. All that talk about players giving up was heavily pointed at Nnamdi, so bringing him back is certainly a knock against the “change the culture” plan. Almost every player is attractive at the right price, including Nnamdi, but I’d rather tag DRC and pay him $10 mil for one year than bring back Nnamdi at half the price (he’s guaranteed $4 mil, so any renegotiation has to be higher or he wouldn’t accept). If they do bring him back, he’s going to need safety help, since it was abundantly clear last year that he doesn’t have the speed to stay with fast WRs.

– Nightmares of the 2009 draft

The general draft discussion seems to have settled on the “deep class but no elites” theme. Allow me to translate: “there are elite players in this class, but we (scouts) have no idea who they are”. This is definitely bad news for the Eagles, as it diminishes the value of their pick, but don’t believe any story that says there aren’t elite players in this year’s draft. The nightmare scenario for teams at the top is a repeat of 2009. Here is the top 15 from that year, with pro bowlers highlighted yellow:

There are a few nice players in there, but the top 5 is a relative wasteland. However, taken later in the first round were Clay Matthews and Percy Harvin. And Shady was available in the second round. While I doubt this year will be as weak as 2009, it is definitely a cautionary comparison as we get closer to the draft.

Rest assured, there will be elite players available at #4. Whether or not the Eagles (or any other team) can identify them is another story.

– Geno Smith

Not a fit for the Eagles, and I’m definitely not a scout, but this guy ran a 4.59 and, more importantly, completed 67% of his college passes. The last time a QB was not selected in the top 5 was 2000, and that may have been because one year earlier 5 QBs were taken in the top 15. If you’re looking for a guy that might sneak into the top of the board (and all Eagles fans should be), this is your best bet.

Luck vs. Skill in the NFL Draft

Posted on February 22, 2013 by Brent

Today I’m going to take a shot at divining the relative importance of luck versus skill in the NFL draft. This is a complicated subject, and as such I’ll probably try a few different methods over the next few weeks.

Before I get started, I’d like to note that this is a different analysis than the previous draft performance evaluation I did. Whereas that attempted to grade teams according to how efficiently they used draft resources, this breakdown is purely about maximizing production.

Ranking the teams:

The first step I took was to go back through each draft and rank team performance each year. I used the Pro-football-reference.com CarAV stat to gauge individual player production, then added each to get a measure of every team’s draft class by year. I then ranked each team according to that production for each year and used those rankings to arrive at an average draft performance ranking. This table should make things a bit clearer:

I know it’s tough to see, but those with good eyesight will find annual draft rankings for each team from 1999-2009. The one big note here is that the CarAv numbers do not include this year’s statistics (I haven’t updated my database yet). So players who broke-out this year will not be fully accounted for. Also, this does not account for different numbers of draft picks, so teams that trade picks for players will be undercounted here. This is ONLY a measure of total production.

As you can hopefully see, the Eagles performed quite well by this measure (as they did in our other draft analysis). Here the team is ranked 8th with an average ranking of 14th. The team’s best draft came in 2002, when the Eagles ranked 1st overall (the Brown, Sheppard, Lewis, Westbrook draft).

Other strong years for the Eagles were 2005, when they ranked 3rd overall (Patterson, Herremans, Cole) and 2009 when the team ranked 5th (Shady, Maclin).

Low points for the Eagles were 2003 and 2004 (ranked 28th and 30th).

Also, notice Super Bowl winners Green Bay, Indy, Baltimore, NYG, and Pittsburgh all placed in the top ten.

FYI, if you are examining the chart, you shouldn’t get too hung up on the 2009 rankings (or even 2008), as those are most subject to change once I add this year’s data.

Luck Vs. Skill:

Now that we have annual rankings, we can look to see if performing well one year gives any indication of performing well the following year. If the draft is mostly skill, then those front offices that are “good” at drafting should be consistently ranked towards the top, with “bad” drafters consistently at the bottom. Theoretically we’d see less consistency among the “bad” drafters since presumably being bad would lead either a change in strategy or a change in decision-makers.

The are some obvious caveats before I get to the numbers. I did not account for changes in front office personnel. The CarAv measure is far from perfect, as we’ve discussed before, so the rankings are bit subjective. Additionally, not every team is trying to maximize production. Some use draft picks to fill roster holes, so judging teams by player production doesn’t give them credit if they have other goals in mind (whether they should have any goal besides maximizing production is another story).

Below is the chart, but let me make something abundantly clear before I show it to you. I AM NOT SUGGESTING THE DRAFT IS COMPLETELY LUCK. As I just mentioned, there are a lot of caveats to this type of breakdown. Also, as I said at the top, there are a number of different ways to look at the subject, so we can’t jump to conclusions from just one.

Still…

Pretty much the definition of no relationship.

I also looked at multi-year averages in an attempt to get rid of some of the noise caused by a random bad/good year. Below is the same chart, using average rankings from 99-01, 02-04,05-07,08-09 instead of individual years.

Yet again, nothing there.

Again, this doesn’t mean there is NO skill in drafting, it’s just a starting point that suggests there MIGHT BE a lot of luck (and certainly a lot more than GMs would like people to think). I’m going to try to account for some of the holes and weaknesses in this analysis, and I’ll repost when I finish. There is a lot of potential noise here.

One more chart:

This is the frequency chart of our rankings. Sure looks a bit Normal doesn’t it? (Normal=random). The standard deviation is 3.37, meaning if it’s normally distributed we would expect about 68% of the values to be between 13-19 (inclusive). In our table, 22 out of 32 are within that range….or 68.75%.

Additionally, we’d expect 95% of the values to fall within 2 standard deviations (outside the 10-22 range in our table). We have 2 30/32…or 93.75%.

What this means (I could be wrong, I’m not a statistician) is that if luck was the sole determining factor for the draft performance rankings, you’d get a distribution that looks A LOT like the one we see above.

Lot’s more to do on this subject, but for now its Luck 1 – Skill 0.

Persistence of sacks

Posted on February 20, 2013 by Brent

Yesterday I mentioned that sack differential and turnover differential have a similar correlation to winning. However, I also said that despite the relatively equal importance of both, teams should focus more on the sack numbers, as that measure is likely much less random (luck-based). Today I wanted to illustrate that:

First we’ll look at the persistence of both sack differential and turnovers. We’ll run the same correlation analysis as we have previously done, this time looking at whether one year’s data point is related to the corresponding data point the following year. If the value is high (absolute value), then the measure is less random, whereas if there is no correlation then it is largely determined by chance (luck).

Here is the chart for sack differential, remember the data set comprises the last 10 seasons for all 32 teams. Note that in this analysis I have not used 2012/2013 pairs because, obviously, that data doesn’t yet exist. The Y-axis is sack differential one year, the X-axis is sack differential the following year.

I’ve included the trend line because without it it’s tough to see the relationship. The correlation value is .29. Not huge, but significant. However, by itself it doesn’t tell us too much. We’ll look at turnover differential persistence now, then we can compare the two.

The value here is .11. So there is some persistence here, but not nearly as much as we saw in the sack differential. This tells us that although both turnovers and sacks are very important when it comes to winning, sacks are much less luck-based, meaning there is a lot more teams can do to effect the measure. Given the choice, teams should look to improve line-play over trying to force/prevent turnovers. (I realize they’re not mutually exclusive concepts, we’re operating in a vacuum here).

A common theme in this blog is how important the efficient allocation of limited resources is when it comes to putting together a football team. It’s vital that teams understand what is luck and what isn’t, so they know where to focus their efforts in order to maximize the impact of those efforts.

I’m assuming little of this is very surprising to readers here. We can, however, take the analysis one step further. Sack differential (and turnover differential) includes both offensive and defensive numbers. What happens if we break them out?

I looked at both sacks and sacks allowed, to see if there was a noticeable difference in persistence. First the charts, then the numbers, then interpretation.

Here are the values:

Sacks: .23

Sacks Allowed: .38

I don’t know about you, but I’m a bit surprised at how large the difference is. There are a TON of variables to consider here. Both sacks and sacks allowed can be greatly effected by the QB, scheme, line-play, schedule, etc… Regardless, the above analysis shows that teams appear to have much greater control over how many sacks they allow as compared to how many sacks they create. This makes some sense, in that the offense has control of every potential sack situation (QBs have a lot more control than defensive players by virtue of having the ball and being able to throw it away.)

In yesterday’s post, I talked about looking at LOS play as a continuum, rather than viewing the OL and DL as discrete units. If we continue with that train of thought, and apply the above analysis, it certainly appears as though teams should pay more attention to the OL than the DL. The goal of preventing sacks on offense is much easier to achieve than the goal of creating them on defense.

Therefore, all other things being equal, I’d much rather have a great OL than a great DL. This has obvious implications for the Eagles. There is a general consensus that the team will be looking at either OT or DT with the #4 pick in the draft. IMO, the analysis here would support leaning towards the OT. Note that we’d still have to evaluate it in the context of the Default Draft Strategy card I put together a few weeks back (which we’ll do once all the scouting shakes out).

For general reference, not everything I show will be counterintuitive or incredibly surprising. I believe it’s important, though, to illustrate concepts that we take for granted or that have a clear logic to them, if for no other reason then to confirm their validity and hopefully get a better sense for their magnitude and/or context.

I’m going to spend most of today and tomorrow in a car, so I probably wont be able to get anything up tomorrow (or respond to comments). Be back on Friday though. Please do comment, as I think that’s extremely important for both developing ideas and challenging my assertions. Not everything I do is right (shocking), but if nobody points it out or argues, I probably won’t realize it.

DRC decision, other thoughts, and a look at Sack Differential

Posted on February 19, 2013 by Brent

Hope everyone enjoyed the holiday weekend. I’ve got some random thoughts to get out before they get too stale, followed by a quick look at the importance of sacks:

DRC: I would have tagged DRC, and it would have been an easy decision. In my mind, the only possible reason for not doing so is if DRC was actively undermining the coaches. I know a lot of people have resorted to calling DRC a “locker room cancer” and such, but I just don’t get the sense he was that big of a problem.

Last year the entire team fell apart. The coaches appeared to be undermining each other. Two were fired midseason and it became clear that the head coach would be let go after the season ended. Is it really a surprise that this led to a team “quitting” on itself? I’m not trying to excuse that type of play, but the fact is almost the entire team was guilty, and it’s reasonable to think that in a better atmosphere, DRC will return to somewhat consistent play.

He was very bad for a lot of last year, and clearly is not worth $10.7 million. However, the Eagles DONT NEED THE CAP SPACE. It really doesn’t matter how much you’re paying him, since it’s only a one year commitment and it doesn’t preclude the team from doing anything else. Worst case scenario: he sucks again and is gone after next year.

As it stands, it looks like the Eagles will not bring him back. That means the team is now looking for an entire starting DB corps. Maybe in a better line-up Nate Allen is acceptable. Maybe in a great line-up Kurt Coleman can be covered for. That still leaves 3 starting spots. I know Howie has claimed to be fully committed to BPA (best player available) in the draft, but it’s going to be extremely tempting to take someone like Dee Milner. If that ends up being a reach, then the Eagles have blown a huge opportunity to kick-start the next phase of the franchise.

If they don’t draft Millner (or another top prospect), where is the team going to find 2 starting CBs? I’ve mentioned before that this is not a 1 year process. Still, I was hoping the team would use its resources to band-aid a few of the holes it couldn’t fix this offseason. DRC fits this role perfectly (band-aid).

Draft Watching: It’s fun to read about all of the draft prospects and compare the “big boards”, but remember, at this point most of it’s useless. The boards are going to change, some dramatically, so don’t get too caught up in things like “Floyd has passed Star” or “no QB’s are worth 1st round picks”. We’ll probably get a very different story closer to draft day, after all the interviews are done and teams have done a lot more film work.

I do like looking at the National Football Posts rankings, though. They claim that their evaluations will not be affected by the combine, only by prospect interviews and character issues from here on in. They’ve got Nassib ranked #1 overall…(he’s the guy the Eagles likely have their eyes on in the Mike Vick-as-insurance scenario). If this ranking holds, safe to say he won’t be there for the Eagles in round 2.

When it gets closer to the draft I’ll pull together a “consensus” prospect ranking, and we can use that with our Draft Strategy Chart to look at what the Eagles should do.

Nick Foles: He will probably be traded. Again, ignore what the “Eagles sources” are saying. Given the supposed weakness in this year’s QB draft class, holding on to Foles until draft day and then auctioning him off will maximize his value. Some team will be desperate for a QB and see their top choice gone. If things fall the right way, the Eagles can definitely get a 2nd round pick for Foles. At the very least, he’s should garner a 3rd + a later round pick (and IMO he’s worth a lot more than that).

OG’s in the first round: I don’t care what Mike Mayock says. I don’t care if Chance Warmack will immediately be the best guard in football. Choosing a Guard in the first round is a dramatic misallocation of resources. The draft is not just about finding prospects who will play. It’s about maximizing the VALUE of every pick.

Sacks: This relationship will be fairly obvious, but it’s not a stat I see referenced very often (sack differential). Obviously getting sacked is bad and sacking the other QB is good, but the overall sack differential (sacks-sacks allowed) is HIGHLY correlated with winning. The only Super Bowl champion in the last 10 years with a negative regular season sack differential were the Ravens this year (-1). Here is the chart, with Sack Differential on the y-axis (and SB winners in yellow):

The correlation value is .62, a very big number. For reference, this is about the same as the value we found for TO margin (.64). However, I’d argue that sacks differential is MUCH easier to effect than TO margin. (I’ll do a persistence post as a follow-up). The idea here is to view OL and DL play as more of a continuum rather than discrete units. Overall, you want to win the LOS (line of scrimmage). That could mean EITHER allowing fewer sacks on offense or creating more on defense (I’m intentionally ignoring the run game here). Ideally you will be good at both, but it’s the net effect that we are looking for.

So for example, rather than fixate on either the OL or DL in the draft, the Eagles should take whoever will add the most to EITHER unit. Right now, NT is a bigger immediate need than anything on the OL, but if the team finds an OT that adds more than the available NT’s, it should draft the OT. Given the choice between a Great OL/Bad DL combo and a decent OL/DL combo, it may in fact be better to go with the first choice, which runs counter to popular draft strategy. Teams typically focus on improving weaknesses, when they should focus on BIGGEST IMPACT. So choosing to take an area of strength and make it dominant (say the 49ers taking an OL) may be a better strategy than trying to go from mediocre to good at another position group. This is of course all relative (and near impossible to accurately quantify), but it’s more support for taking the BPA rather than need.

Eagles Rewind

Objective and Analytical Analysis of The Philadelphia Eagles and the NFL