Prospect Rankings via Positional Value Multiplier

Today, with the help of a collaborator, I’ll give you prospect rankings for the NFL Draft that you won’t find anywhere else.  As I’ve explained before, I am not a scout and have not watched film on every top prospect in this year’s draft. However, I believe what I’ll show you today is more useful than any individual scout’s ranking.

First things first, big thanks to George Laevsky (JD from Georgetown) for the help.  He came up with the idea and name for the Positional Value Multiplier and worked with me on compiling/computing the necessary data.

To keep this clean, I’ll explain it in 3 sections.  First I’ll tell you what we did, then I’ll tell you how we did it, then I’ll show you the results.  That way, if you want to skip the middle section you can.

What We Did:

The overall aim of this project was to apply a positional value modifier to the consensus prospect rankings, with the hopes of generating a more accurate system of ranking value.  We compiled a composite prospect rating for each player (through the first couple rounds) and then adjusted for positional importance according to last seasons’ league-wide positional salary distribution.

Before we go into the How details, here is the consensus prospect ranking using ratings from Scouts Inc (ESPN), the National Football Post, and  Note: NFP uses a different grading scale, so those scores were adjusted to give us an apples-to-apples rating.

Screen Shot 2013-03-07 at 11.19.43 AMThat graphic alone is pretty interesting, particularly when the ratings diverge (see Ryan Nassib at the bottom), but we’ll look at that some other time.

For today’s post, we have to adjust.

How We Did It:

I mentioned last week that no BPA ranking is complete without an adjustment for relative positional value.  For example (an extreme one), if a QB and K both carry a 95 rating, you’d obviously choose the QB first.  The question is, how do we measure relative importance by position?

While there is no bullet-proof method of doing so, the salary distribution in the NFL is as good a place as any to divine information from.  In theory, since the NFL has a salary cap, the distribution of limited funds between positions will give us an idea of how the league, on average, values different positions in relation to each other.

We pulled salary cap information from this awesome graphic featured in the Guardian at the end of January.  It’s not perfect (reflects cap hits from last season and misses some IR guys), but in general I believe it’s as good a breakdown as any for our purposes today.  After adjusting for the number of players by position, we calculated a Positional Value Multiplier (“PVM”) for each major position (FB, K, P not included).  We then applied that multiplier to the above consensus rankings.

Here are the multiplier values we arrived at, in order from largest to smallest:

Screen Shot 2013-03-07 at 11.26.43 AM

For the most part these make a lot of sense, based on what’s “common knowledge”.  QBs are, by far, the most important position.  However, the relative rankings of WRs and RBs certainly surprised me, though due to the noise in the data, it’s best not to get hung up on the minute differences in values above.  Instead, we can see there are some clear “tiers” (I feel like I am using that term a lot).

Tier 1 – QBs

Tier 2 – WR, CB, DE, RB, DT

Tier 3 – OT, LB, TE, S

Tier 4 – C, G

The only thing in those rankings that immediately draws my attention is the OT position in the 3rd tier.  But that data is what it is, we can debate the reasons later.

Now that we have the PVM values, we can apply it to the prospect rankings.

The Results:

Screen Shot 2013-03-07 at 11.42.44 AM

Some very interesting movement.  The right-most column shows the effects of the positional modifier.  The AG Rank column is the pre-adjustment consensus ranking.


– Dee Milliner jumps two places to become the top overall prospect.

– Chance Warmack, though he drops 3 spots, remains a top 5 prospect, damaging my belief that a G in the top 15 picks is a very poor decision.

– QBs, as expected, benefit the most.  Geno Smith jumps 11 spots to become a top ten prospect, while Nassib and Barkley move into the middle of the first round.

– The biggest jump overall comes from Tyler Wilson (20 spots), who moves from the middle of the second round to the end of the first.

– Zach Ertz (TE) and Jonathan Cyprien (S) are hurt the most, falling out of the first round, and therefore off the chart above.

One last thing: I want to be perfectly clear about the value of this analysis.  The idea here is that BPA is an overly simplistic and flawed method of drafting by its current definition.  For example, while Geno Smith (19th consensus) may be a worse prospect than Kenny Vaccaro (9th consensus), with lower odds of success, the potential payoff is so much greater for Smith that he becomes a better choice (at least as shown here).  Hitting on a QB offers a MUCH greater reward than hitting on a S (or really any other position), so it makes complete sense that QBs are perennially “over-drafted”.

In essence, what we are showing here is that they are not, in fact, “over-drafted”.  Yes, they might have greater odds of failure, but that does not make them bad picks.  Remember, you have to look at both Risk AND Reward, balancing the two.  The above rankings is an effort to do that in a method as simple and transparent as possible.

Over the past few months, I’ve tried to advance the idea that the “consensus” forecast should carry a large degree of inertia within NFL front offices.  Imagine the above rankings as equivalent to a total market stock index.  For anyone going against the total market index, they must believe VERY STRONGLY that they have better information or better analysis than the rest of the market.  It should function in much the same way in the NFL (and all professional sports leagues).  The idea is NOT that teams should blindly follow the “market”, just that they should hold their own evaluations up to very high scrutiny before acting on them, especially when they largely conflict with available data.

I’ll be examining this in a lot more detail, which may or may not lead to more posts on the subject.  In any case, this should give everyone something to think about come draft day.

For what it’s worth, my subjective pick for the Eagles would still be Lotulelei/Joeckel.  However, unless I adjust the PVM formula (or if the consensus ratings change), it looks like Dee Millner is now, objectively, in the lead.

UPDATED: Also, below is the same analysis for the rest of the players we looked at:

Screen Shot 2013-03-07 at 3.06.03 PM

Screen Shot 2013-03-07 at 3.08.36 PM




BPA absent Positional Value adjustments

Last post I attempted to explain, in no uncertain terms, why taking a Guard in the first round makes little sense (especially in the top half of the first round).  However, I get the sense that many people remain unconvinced.  Today, I’ll take a slightly different look at it to see if I can sway the remaining holdouts.  I apologize for the repetition in subject-matter, but this is an EXTREMELY important topic, as it goes to the heart of Optimal Draft Strategy.

First, I want you to ask yourselves if you agree with the following statement:

GMs should, in general, take the Best Player Available (“BPA”) with every draft pick.

I’m guessing most readers here would back that strategy, and I certainly do.  HOWEVER, it is not enough to just endorse that statement and use it as the basis for a draft strategy.  First, you must definite exactly what BPA means.

Here is where I am seeing some confusion and where I believe there is a big disagreement.  The people calling for taking Warmack high in the draft (forget about the Eagles for a moment, I’m speaking in broader terms), are using BPA as support.  The problem is that, in their BPA definition, they do not seem to be adjusting for positional value, which is a MASSIVE mistake.

Rather than attempting to explain why with a rational argument (for that see my last post), I’ll try to illustrate it.  Here are the top ten OTs and Gs from this past season, according to Pro Football Focus.  They are listed in order, with their original draft pick included.

Screen Shot 2013-03-04 at 11.34.40 AM

There are two big takeaways from this chart.  First, look at the draft picks.

I know I’ve said this repeatedly, but if you want an “elite” OT, you have to take him in the first round.  The same clearly does not hold for the Guard position.  This provides a clearer illustration of the opportunity cost argument I made last week.  There are really only two ways to get an elite OT in the NFL, draft him or sign him in FA. (I know we traded for Peters, but that’s a RARE exception).  Additionally, signing an elite OT in FA requires a huge contract.  Therefore, the BEST way to add a top OT to your team is to draft him in the first round.  Choosing a G means you are forgoing that opportunity.

The second point I want to highlight from the chart above gets at positional value.  Again, I want you to answer a question:

All other things being equal, would you rather trade Anthony Davis for Evan Mathis (or any other top Guard) or vice-versa?

I’m a big Mathis fan, but that’s a no-brainer.  Clearly most of the league agrees as well, hence the salary differences between OTs and Gs.

Assuming you agree, that means the #10 OT is worth more than the #1 G.

How about Brandon Albert versus Evan Mathis?

This one probably creates a bit more disagreement, but my guess is, on the whole, people would take Albert, who ranked as the 25th best OT according to Pro Football Focus.

Regardless, clearly there is a discrepancy in value between OTs and Gs.  Also, while I’ve focused on Gs and OTs, the same analysis can (and must be) done with all positions, and then incorporated into each team’s prospect rankings.  To not account for this is a huge mistake and one that appears is made by a lot of the writers who support taking a G high in the draft under a flawed concept of BPA.

To make sure I’m being clear: This should be incorporated into the “tiered” rankings I advocated previously.  So it is theoretically possible for a G to be the best choice with a top 15 pick, but it would require an amazing G prospect that can overcome the positional value difference to make it to the top 1 or 2 tiers, as well as every other player (from positions of greater value) within the same prospect tier being taken before said top 15 pick.

Hopefully this has convinced a few more of you, or at least provided a clearer explanation of what I was getting at last week.  This isn’t exactly Eagles-specific, because I don’t think there’s any way they take Warmack at #4, but it applies to any team with a top 15 pick.

Lastly, I’ll leave you with an analogy.  A frequent defense of G picks (and OL picks in general) in the first round has been that they are “safer”.  In general, that is true, as our strategy chart showed.  However, this is only one half of the equation.  As anyone in finance knows, evaluation is always a question of risk vs. REWARD.  To look at one side and ignore the other is a recipe for sub-optimal decision-making.

For example, given the choice between to raffles, and ignoring an external factors, which would you rather enter:

A) 60% chance of winning $100.

B) 75% chance of winning $75.

While the odds of success for raffle B are significantly higher, the correct answer (in a vacuum) is A, since the expected payout is greater.  It’s impossible to apply such specific values to players, but it’s a useful exercise nonetheless.  Here, imagine a DE as raffle A and an OL as raffle B.  To take the OL just because he offers less risk (25% chance of miss versus 40%) would clearly be a poor decision.