The Rules for Drafting (Abbreviated)

Today’s the day.  Let’s go over the ground rules and lay out a few key things to watch for.

The Rules

1) Do not fall in love.  Trading up for a player or reaching for someone is a surefire way to blow things.  Regardless of how confident you are in a certain player, we know that there is a significant chance that you’re wrong.  Failing to understand the uncertainty in your own evaluations/models is a killer, no matter what industry you’re in.

2) If a players “falls”, there’s usually a reason for it.  A “reach” pick, though, is usually just a bad pick.  This ties into the first point.   Below is a chart showing the biggest “reaches”, as I’ve defined them (difference from TPR), from 2010:

Screen Shot 2014-05-08 at 3.52.27 PM

Some of those guys are good players.  Most of them are not.  Not only did most of these players “bust”, but they probably cost the teams that chose them more than they should have.  If there’s a player you really like, trade down until he makes at least some sense at that draft spot.  Chances are you are not smarter than the rest of the league.

3) The 6th – 7th round arbitrage.  Always trade down in the 6th round.  Look at these success rates:

Screen Shot 2014-05-08 at 3.56.33 PMScreen Shot 2014-05-08 at 3.55.54 PM

Do you see the difference in success rates between the 6th and 7th round?  No?  That’s because there isn’t much difference.  Think about every draft pick as a lottery ticket.  The 1st round picks have the highest probability of “hitting” and those odds decline as you progress through the draft.  BUT, once you get to a certain point in the draft (somewhere around the 6th round), the odds flatten out.  At that point, everyone has roughly the same (low) odds of becoming a starter in the league.

I realize that every team gets to the 6th round and still has “fifth” round guys or better on their board.  That’s where self-awareness comes into play.  That late in the draft, NOBODY KNOWS ANYTHING.  If the player of interest was that good, he wouldn’t still be there.  As far as “free lunches” in the NFL Draft go, this is the most obvious.  Trading a 6th for two 7th round picks essentially doubles your odds of success.  It’s a no-brainer, and will be available for as long as there are over-confident GMs (so forever).

There’s obviously a lot more, but I’ve gone over most of it in the past and let’s face it, you won’t remember more than 2-3 of these anyway.

Things to Look For

– Teddy Bridgewater, Dee Ford, Ra’shede Hageman, Ryan Shazier, Eric Ebron Bradley Roby.  If you’re an Eagles fan, you should be hoping to hear these names called before #22.  As I see it,

Plan A for the Eagles is Mosley or Barr,

Plan B is Pryor or Clinton-Dix,

For one of those guy to all, we need some of the guys I listed to go earlier than expected.  Also watch the WRs and QBs.  A run on either of them is very good for the Eagles.

– Watch the QBs.  Last year, the top QB prospects, according to the “experts”, fell an astonishing distance in the draft.  This might be an anomaly.  Perhaps there’s information the teams had that the draftniks didn’t, in which case those guys didn’t really “fall”.  However, it’s also possible that GMs have adjusted to the increasingly passing-dependant league by holding QB prospects up to higher standards.  Maybe GMs, in general, are not as willing to overlook flaws at the QB position like they have in the past.  If so, that’s BAD for the Eagles, and we’ll see guys like Bortles, Bridgwater, etc.. fall.  Paradigm shifts are very rare, but that doesn’t mean we shouldn’t keep an eye out for them either.

– Trade down scenarios.  It’s really anticlimactic, especially when the Eagles have such a late pick, but a trade-down might be the best outcome for tonight.  Like I said, Plan A and B involve guys slipping a bit.  That’s unlikely.  Failing a shot at a top LB or S, moving back and grabbing another pick or two looks like a good plan.  As I showed earlier this week, the WR available at 54 will likely be nearly as good as one available at 22.  Additionally, while there are just 15 players rated 80-90 in my TPR rankings, there are 53 rated between 70 and 80.  Picking up another 2nd round pick would be well worth a trade-down if the defensive impact players are off the board.

– Trade up scenarios.  Lot of talk, and I hope that’s all it is.  If the Eagles do trade up, it means the probably made a strategic error (maybe a big one), but it DOESNT mean the draft’s a failure.  Trading up CAN be the right move, it just requires a degree of confidence that should be very rare.  But “very rare” does not mean “never”.  It’s possible, and if the Eagles trade up, I’ll obviously root for that to be the case.

– Lastly, Chip should have an advantage for this year and next.  Chip recruited and scouted a lot of these players.  This is the type of inside knowledge that couldlead to the type of confidence I talked about above.  If Chip can leverage his knowledge in the draft this year, it’ll go a long way towards building the foundation of a team that can contend for a long time.

That’s all.  It’s been a ridiculously long wait, but the draft is finally here.  Enjoy it.

Hacking the Draft

For those of you who don’t remember, last season I did a historical analysis of draft pick success according to position and round.  Below is one of the posts stemming from that project.  At the bottom are two charts showing round-by-round probabilities.   Note: counting someone as a “starter” is a bit subjective.  Inclusion in the below data means that a player started for at least 5 years in the league (according to Pro-Football-Reference), or if they joined the league less than 5 years ago, has started for more than half the time.  The sample is all players drafted between 1999 and 2011.

Hopefully everyone has enjoyed the round-by-round breakdown.  While there are obviously a number of variables that can’t be controlled for, the pure statistical look at each position group has already provided some interesting insight.

Before I get to the big chart, let me just clarify exactly what I think this type of analysis is good for, then dig into an Eagles example.  Feel free to skip to the chart and come back.

The purpose of this is NOT to arrive at a set of rules by which teams should draft players.  Instead, it is meant to provide a general guideline, or a “default draft position”.  For example, yesterday I said that selecting DEs in the 3rd round is a terrible value proposition according to the data I’ve collected.  That doesn’t mean drafting a DE in the 3rd round is ALWAYS a terrible decision, it just means that for a team to make that decision, it must see or know something about the subject player that makes him CLEARLY much better than any other prospect available at that position.

The biggest flaw in NFL draft strategy, as far as I can tell, is each team’s confidence in its own ability to evaluate talent.  Regardless of the general manager, every team has historically had a very large margin of error when it comes to talent evaluation.  For instance, in the 1st round less than 6 in 10 LBs selected from 1999-2011 had or are having significantly productive careers (according to our definition.)

So what does that mean?  It means teams, in general, should be mostly focused on value during the draft, as opposed to parsing prospects.  This is perhaps never more apparent than when a team decides to trade up to select someone.  Let’s use the Eagles as an example.  NOTE: This is a very rough example, with numbers pulled from my ass, and is only meant to illustrate a larger point.

When the team traded up for Brandon Graham, it swapped 1st round picks with Denver and gave up two 3rd rounders as well (moving up from 24 to 13).

At the 13th pick, no DEs had been taken.  Between picks 13 and 24, 3 were taken, including Graham.  So that means the Eagles, in their analysis, decided that they had to take a DE (will not argue that decision here, though I was mad they passed on Earl Thomas, and can produce witnesses that will verify I said that when it happened).

The only way the trade made sense was if the Eagles, in their DE analysis, decided that the odds of Graham becoming a stud DE were MUCH higher than the odds of JPP, Morgan, or Hughes becoming a stud DE.  Here is where the “margin of evaluation error” comes into play.

Using our historical draft data, we can calculate the odds of getting a starting DE with a 1st round pick and two 3rd round picks (I realize they were hoping more than an average starter, but stay with me).  Using the table below, we can calculate those odds to be 81.5%.  Using the Pro Bowl percentages from the earlier tables, we arrive at 37% for the odds of getting a Pro Bowl DE if you select DEs with a 1st round pick and 2 third round picks.  So here is the breakdown for “generic DEs”:

– 1st Round Pick – 24% chance of Pro Bowl, 67% chance of starting

– 1st Round Pick and 2 third round picks – 37% Pro Bowl, 81.5% starting.

Please note that this DOES NOT mean the Eagles made a bad decision.  Obviously the odds should be better for the 13th pick than for the 24th pick (we’ll get to that another time).  It DOES MEAN, however, that the Eagles, in their evaluation of Brandon Graham, should have been almost certain that he was more than 37% likely to be Pro-Bowl caliber, and more than 81.5% likely to start.

I have no doubt that they believed this, BUT, if they had applied a margin of error to their own analysis (as any good team should) prior to making the trade, they would have been unlikely to go through with it.  Let’s be extremely generous and assume the Eagles front office could peg these odds with a MOE of +/- 15% (WARNING: overly simplified statistics).   That means if they estimated Graham had a 90% chance of starting, his true odds of starting were almost definitely between 75-100%.

Immediately we can see an issue.  Even giving Graham incredibly high odds of starting (90%, which is more bullish than any team should be with any players outside the top few picks) and the Eagles a very generous MOE (+/-15%), the resulting range still does not exclude the 81.5% starting odds for the generic position of a first and two thirds (although it is near the bottom of that range).

That means the Eagles really should NOT have been (though I’m sure they were) confident that Graham would be better than just taking whichever DEs were available at #24 and in the third round, and hence, should not have made the trade.

I don’t mean to suggest that trading up is never a good idea, simply that the evaluative bar for whichever prospect is the target must be EXTREMELY high, and much higher than the standard currently being applied by most teams.

My final point:  Teams do not appear to take a probabilistic approach to drafting (which they almost certainly should), and I would guess that they do not actively overlay a margin of error onto their evaluations.  This is very much a “new-school vs. old-school” issue, similar to the statistical revolution in baseball, but IT IS NOT THE SAME.  It is largely a matter of GMs being willing to recognize and account for their own shortcomings and cognitive biases.  The franchises that can apply this will, in the long run, be more successful than those that don’t.  (Looking into which teams might be using this type of strategy will be another day’s post)

Ok, enough talk.  Here is the chart with every round included. I removed the All-Pro and Pro Bowl columns to make it easier to compare.  Dig into it and see what you think.  Later this week we’ll mine it for an “optimal default strategy”.

I updated the chart soon after the original post,  here is the updated version; so the numbers might not match the post exactly.  The original is reproduced below.

Screen Shot 2014-05-07 at 10.29.30 AM

Screen Shot 2013-01-29 at 10.54.06 AM

Examining the WR prospect tiers

There’s a lot of talk about the Eagles potentially trading up for a WR, or at least taking one with their 1st round pick.  Peter King has them giving up their 1st and 2nd round pick to get Odell Beckham Jr.  I’ve been very clear about why I think this is a poor strategy (trading UP for a WR is an unbelievably bad decision).  For more on those reasons, see my last post.  Today, though, I wanted to look at it from a different angle and discuss things in light of what the actual WR class looks like.  Previously, I left it at “it’s deep”, which doesn’t really provide the full context.

From my TPR rankings, here are the top 10 WRs in the draft.  Remember, since each of these guys play the same position and I took individual standard deviation out of the formula, these relative rankings are purely rankings are primarily the result of the NFL.com, ESPN, and NFP grades.  The multiplier stratifies the class a bit, but the effect is small.

Screen Shot 2014-05-06 at 10.04.32 AM

We can ignore Sammy Watkins and Mike Evans.  I’m sure Chip would love to grab Evans, but it’s extremely unlikely he falls out of the first 10 picks.

Instead, let’s focus on the second tier, the yellow shaded area.  Here we have four WRs that have all, at various points in time, been linked to the Eagles.  If the Eagles come out of the 1st round with a WR, it’s nearly a certainty that it’s one of these four guys.  But that’s not what we’re here for, is it?

Looks closely at each of those prospects and look at the grade assigned.  Now, how certain are you that you can identify which one will be the best NFL player?   “Not at all” is the correct answer.  I’m sure the Eagles have different grades and a different order of players, but the fundamentals are the same.  You need to ask yourself two questions:  How big is the difference between each prospect’s grade?  How big is the margin of error in our evaluations?

Within each tier, the MOE (if you’re being honest) is almost definitely larger than the difference in grades.  Therefore, practically speaking, they all have the same grade.  I other words, they all have the same expected value.

So why would you want to pay more for one of them than they other?

Now, let’s take aim at Peter King’s rumor, which is:

Eagles trade up for the 15th pick and select Beckham.

First, let’s see how necessary that trade is.  If they complete it, obviously they get Beckham.  If not, though, how likely is he to be available at the 22nd pick?

Well it just so happens that Brian Burke of advancedfootballanalytics.com (new name) has just released a Bayesian prediction model for the draft.  Obviously, we can’t put too much weight into this just yet, but it’s a very good representation of the type of thinking every team should be doing.  Here is Beckham’s output:

Screen Shot 2014-05-06 at 10.25.12 AM

According to this model, there is just a 7% chance that Beckham is available at the 22nd pick.  There is a 53% chance he is available at the 15th pick.  The Eagles would only make the trade if he was actually available at 15, so we don’t have to worry about that second probability.  Just note that it’s basically a flag that says: even if the Eagles and Steelers want to make that trade, there’s a 50/50 chance it won’t happen.

So, if the Eagles want Beckham, it looks like they really do need to move up.  How about the other guys in that tier?

There is a 51% chance that Marqise Lee will be available at 22.

There is a 21% chance that Brandon Cooks will be there at 22.

There is a 92% chance that Kelvin Benjamin is there at 22.

Now, these aren’t completely independent probabilities, so what I’m about to do isn’t 100% “correct”, but it’s not unreasonable either.  Combining those probabilities leaves us with a:

.49 * .79 * .08 = .03 or 3% chance that none are available.

So, the Eagles can give up their 2nd round pick for a 100% chance of Beckham, or they can keep their pick and have a 97% chance at one of the other three guys in that same tier.

Now can you see why trading up is such a terrible value?  We’ve already covered the margin of error issues.  Regardless of which players are in the same tier, conceptually they are all worth the same “value”.  So if the Eagles tiers looked like mine, they’d essentially be trading a 2nd round pick for a 3% increase in the odds of getting a WR from their desired tier.

That’s also known as a catastrophically bad use of resources.

Now let’s look at it a little differently.  Let’s say the Eagles do have Beckham rated significantly higher than the other three guys in that tier.  The operative question then becomes: how much higher?

This is important because we have to account for the opportunity cost of the 2nd round pick (which is large).  That brings me to the concept of saturation drafting.  In short, there’s no rule against using multiple picks in one draft on the same position.  For example, let’s say the Eagles have decided they NEED a star WR out of this draft.  They can:

A) Do Peter Kings trade, after which their odds of gaining a star WR will be whatever the odds of Beckham becoming one are.

OR

B) They can NOT trade their 2nd round pick, and use it on ANOTHER WR!

To examine this possibility we need to know who will be available in round 2.  Let’s move to the next tier on our list.  This one:

Screen Shot 2014-05-06 at 10.47.24 AM

What are the odds those players are available at the 54th pick?

Davonte Adams – 38%

Jarvis Landry – 62%

Cody Latimer – 8%

Jordan Matthews – 37%

Once again, they’re all in the same tier, and the individual margin of error means they each of roughly the same expected value.  Given the odds above, that means there is a:

.62 * .38 * .92 * .63 = .136  that none of those guys are available at 54.  Flipped around, that means there is a 86% chance one of those players will be available at 54.

Going back to our two options, that means the Eagles, spending the same amount of draft resources, can have:

100% chance of Beckham

OR

97% chance at a player like Marqise Lee AND an 86% chance at a player like Jordan Matthews.

Now, if you NEEDED a star WR, would you choose option A or B?  That ignores a lot of other options (for example, you could trade up in round 2 to give yourself a 100% chance of a 3rd tier WR), but it lays out the conceptual problems with trading up for a WR in round 1.

That’s a very long way of saying what I’ve said before:  If you are going to trade up for anyone, ESPECIALLY a WR, you need to be extremely confident he’s much better than the next guy.  Realistically, I just don’t see how the Eagles could possibly be that confident.

Therefore, trading up for a WR is a very poor strategic decision.  Remember, you’re not picking players, you’re picking lottery tickets.  Each one carries a different likelihood of “hitting”, but they all have risk of busting.  All you’re trying to do is maximize your odds.

The Eagles rankings undoubtedly look different from the tiers I’ve used above, but it really doesn’t matter what names you put in which tier.  Unless the Eagles think the gap between Beckham (or whomever) is EXTREMELY LARGE, trading up doesn’t make any sense.

Lastly, I’ll leave you with some spider graphs (from mockdraftable.com):Screen Shot 2014-05-06 at 11.01.38 AMScreen Shot 2014-05-06 at 11.02.14 AM

Screen Shot 2014-05-06 at 11.03.16 AMScreen Shot 2014-05-06 at 11.03.55 AM

Screen Shot 2014-05-06 at 11.05.11 AM

 

 

Projecting the Eagles Pick

The draft starts this Thursday, which means we have about as much information as we are going to get regarding scouting reports, projections, team interests, etc…  Normally, I don’t do much projecting for the draft. I’ve been pretty clear in the past regarding optimal draft strategy (i’ll try to repost those this week), and the major takeaway is: Do Not Fall In Love With One Player.   That’s arguable the single most destructive thing a team can do.  Nobody’s projections are perfect.  In fact, they’re so far from perfect that the margin of evaluative error likely outweighs the difference between most (or all) prospects within a given tier.

However, I do want to point everyone to a post over at BGN.  James Keane took my TPR rankings and a set of team needs compiled by SBNation local sites and created this probability-driven simulation for the draft.  The results are pretty informative, but they’re not perfect.  I encourage you to read the post and play around with the simulation (it’s interactive), but I’ll cut to the headline.  Here are the “most likely” players to be drafted by the Eagles with their 1st round pick:

Screen Shot 2014-05-04 at 4.37.31 PM

Now about that “not perfect” part.  The biggest issue I see with the above list is Ryan Shazier.  I don’t think he’s a legitimate target for the Eagles.  Unfortunately, the system (my rankings and team needs) don’t allow for the resolution needed to differentiate between weak-side OLBs and strong-side OLBs.  Shazier looks to be a good fit at the WILL spot in a 3-4 defense.  Of course, the Eagles already have Mychal Kendricks there.  I haven’t done any scouting, so perhaps Shazier can be a pass-rushing 3-4 OLB (Trent Cole’s spot), but I doubt it.

Removing him from the list, we’re left with:  Marquise Lee (WR), Calvin Pryor (S), Odell Beckham (WR), Ha-ha Clinton-Dix (S), and Anthony Barr (LB) followed by two CBs.

As much as I disagree with the idea of selecting a WR in the first round, that seems like a pretty good projection given what we know about the Eagles and Chip Kelly.  So those are the guys to watch if you’re looking to do some pre-draft research.

Now, a note on WRs.  Here’s why I don’t like the idea of taking one in the first round:

– The Eagles just cut D-Jax.  Many are suggesting that the team now NEEDs a new WR weapon.  I think that’s backwards.  Ideally, the Eagles would have felt comfortable cutting Jackson precisely because they felt like they DIDNT need him.  If the Eagles end up using a 1st round pick on a WR, then the opportunity cost of the D-Jax move skyrockets.

– The WR position is widely considered the “deepest” in the draft.  If that’s the case, why take one early?  If a position is truly “deep”, then it means the best value MUST lie in selecting one relatively later in the draft.  “Deep” means there are a lot of high-quality players and that the difference between the best guy and the 7th best guy (just throwing numbers in there) is smaller than normal.  Hence, taking one of the first guys is a relatively poor value.

– Lastly, Wide Receivers have the highest miss rate of any position in the 1st round, according to the research I did for the Hack the Draft post.  In other words, WRs carry the highest inherent risk.  Confidence in an individual player can not mitigate the systematic risk inherent in that position.  When you combine that with the relatively small contributions rookie WRs make, it seems like a poor strategy for a team that still has holes throughout the roster.

Officially, I’m just rooting for a value-maximizing pick that adheres roughly to the TPR rankings I developed.  Personally, though, I’d love it if that pick happened to be a Safety (as I’m sure most of you would).

Or maybe they’ll just trade for Johnny Manziel…

2014 TPR Rankings

They’re here.  Below you’ll find the TPR rankings for the 2014 draft.  Scroll down if you don’t just want to see the numbers, but there are a few things we have to discuss before getting there.

As you know from my most recent posts, I’ve had to change the formula this year.  Part of that was by design; I always intended to tweak it over time in order to provide a better representation of what I think is, conceptually, the right way to think about the draft.  So what’s different this year?

I’ve dropped the individual standard deviation.  In theory, I still like this variable as a proxy for each prospect’s individual risk.  If scouts widely agree on a prospect’s ability (i.e. rating), that prospect should represent less risk than a prospect for whom there is wide disagreement.  Unfortunately, the change in grading scales has made using StDev much more complicated.  That, combined with the fact that I only have three different scouting ratings for each prospect, means StDev, as I can currently calculate it, is probably not a reliable proxy for idiosyncratic risk.  However, I still hope to use it in the positional ranking charts.

– Without StDev, we’re left with two factors: Systematic Risk and Positional Value.  This year, I’ve combined them into one factor, an overall Risk/Reward Multiplier.  Our stand-in for Systematic Risk is the Positional Hit rate I calculated for each position when I put together my “Hack the Draft” cheat sheet.  I’ve taken that measure, and multiplied it by the PVM values I posted yesterday.  Basically, we’ve now got an “Expected Positional Bonus” that combines positional impact with historical positional risk.  Make sense?

Just to recap, our formula now looks like this:

Consensus Rating * (1 + (Positional Hit Rate * PVM))

Also, I want to remind everyone what the goal is.  We are NOT trying to predict the draft.  Every team has it’s own scheme, for which different players fit and do not fit.  Additionally, while Relative Value (BPA+) is the best drafting strategy, you can’t be 100% blind to need.  For example, the Seahawks aren’t taking a QB in the first round.  The rankings below tell us, in a probabilistic sense, what the default ranking of prospects SHOULD be if we wiped everyone’s roster clean.

In other words, if the league was starting from scratch, this is how I would rank the players.  Note that I’ve shown each factor.  Miss rate does indeed show the miss rate by position.  I’ve adjusted that in the calc (to flip it to Hit Rate), so it’s not a mislabel or mistake.  Also, for the consensus ratings, if NFP did not rate a player, I plugged in a score of 5 for him (which is a very bad score).  NFP claims to not rank anyone they rate as 4.9 or below.  In the far right column, you can see the players for whom the Risk/Reward adjustment made the biggest difference; a positive number means they were helped by the adjustment.

Here’s the top 150, sorted by TPR Rank:

Screen Shot 2014-04-28 at 9.25.57 AMScreen Shot 2014-04-28 at 9.28.48 AMScreen Shot 2014-04-28 at 5.56.38 PMScreen Shot 2014-04-28 at 9.29.59 AMScreen Shot 2014-04-28 at 9.30.54 AMScreen Shot 2014-04-28 at 9.31.55 AM

 

2014 Positional Value Multipliers

As promised, I’ve calculated new positional multipliers for 2014.  Obviously, the main purpose of deriving these values is to apply them to our Consensus Ratings.  However, before I did that, I wanted to give the Multipliers their own post, because they show slight differences from last year.

As a refresher: The general idea of the multipliers is to properly account for the differing impacts of each position.  For example, a QB has a much bigger impact on the game than a Center does.  Therefore, all other things being equal, the QB prospect will be of more value than the C prospect.  The question is, how do we account for this?

Fortunately, the NFL has a salary cap, which means the league has done our work for us.  Since each team has finite resources, the way they distribute their cap space can, in theory, tell us how they value each position.  Looking at the league numbers in aggregate, we can get a sense of the default relative value the league places on each position.  However, it’s not as easy as just taking an average of every salary at each position.

We’re mostly concerned with the top of the draft (first 3 rounds), where teams are, hopefully, trying to identify and select future starters and stars.  Additionally, we have to account for positional scarcity.  There are a lot more CB snaps than QB snaps, by virtue of two CBs being on the field for every offensive play.  So, here’s what I did:

1) Used Pro Football Focus to determine how many players at each position played at least 50% of their team’s total snaps.

2) I then divided each of those measures by 4.  Basically, this tells me how many players are in the top 25% at each position.

3) Then I simply took the average of every cap hit that fell within that measure.

For example, 28 QBs played at least 50% of their team’s offensive snaps last year.  Divided by 4, that means 7 QBs comprise the top 25% of starting QBs (roughly speaking). The average of the top 7 QB cap hits this year is $18,407,457.

Pretty straightforward.  I did that for each position, then used their relative values to arrive at the PVM values listed below.  Next to them are last year’s values, along with the change year-over-year.

Screen Shot 2014-04-26 at 11.26.30 PM

 

The biggest gain in value was at the Center position.  The multiplier for Centers, while still relatively small, is 2.87% higher than it was last year.  Conversely, both CBs and RBs have seen their values decline somewhat substantially.

Perhaps most interesting is the placement of Safeties.  Despite the conventional wisdom being that Safeties are increasing in importance, the values above show they’re actually worth LESS than they were last season.

The top four spots remained the same (QB, DE, DT, WR), but after that, the order shuffled quite a bit.  However, note the spread of values between the 2nd highest and the lowest for both years.  As you can see, this year there’s a much narrower range.  With just two years of data, we can’t draw any substantial conclusions, but it’s certainly something to keep an eye on.

Next up, we’ll slap the Consensus Ratings and the PVM values together, along with our Risk factors (which I’m changing slightly), and arrive at our PVM Big Board…finally.  Sorry for getting this up so late in the game.  Turns out the NFL’s new draft schedule coincides precisely with law school finals.

Consensus Ratings

I’m compiling this year’s PVM Rankings, and the first step is to compile the consensus scouting ratings.  If you remember, I take the average of each prospects individual rating from ESPN (Scouts, Inc.), NFL.com, and the National Football Post.  I use this as a proxy for prospect quality, and then apply the additional bonus and risk factors I’ve explained before.

Anyway, NFL.com and NFP are actively trying to destroy this exercise.

Check out the table below.  I’ve taken the top 20 players from last year and lined them up with the top 20 players this year (by Consensus Rating).

Screen Shot 2014-04-24 at 9.56.16 PM

The reason for the significant difference is that both NFL.com and NFP have changed their grading scales.  Not only did the site switch to a 10 point scale, but it looks like the relative value changed as well.  That means that even after adjusting for the new scale, the ratings aren’t directly comparable. However, even accounting for that difference, it looks like ratings are lower across the board.  NFP did not make any adjustments that I can see, it just rates this year’s players as worse prospects.  (As always, please correct me if I’m wrong.)

Last year, NFL.com’s ratings were out of 100; this year they’re out of 10.  Previously, the 85-95 rating tier was defined as: “Immediate Starter – An impact player with the ability/intangibles to become a Pro Bowl player.  Expect to start immediately except in a unique situation (i.e. behind a veteran starter).

This year, the top tier, 9-10 is defined as: “Once-in-a-lifetime Player”.  8-9 is now defined as: “Perennial All-Pro”.

NFP has made a similar switch.

As you can see, this is a fairly dramatic departure.  The upshot, unfortunately, is that we won’t be able to compare this year’s draft class to last year.  It’s possible that both of these organizations believe the new systems are superior to the ones they used last year.  It’s also possible that they’ve realized changing the scales makes it really hard to critique the ratings over time.

Whatever the cause, the result sucks for us.  It shouldn’t hurt this year specifically (and may help if the new systems are, in fact, superior), but we can no longer compare players year-over-year to answer questions like: Where would Jadeveon Clowney rank compared to last year’s prospects?

Still working through potential adjustments, but for now, here are this year’s top prospects, by consensus rating (as I’m currently calculating it, which is just an average).

Screen Shot 2014-04-24 at 9.57.03 PM