BPA absent Positional Value adjustments

Last post I attempted to explain, in no uncertain terms, why taking a Guard in the first round makes little sense (especially in the top half of the first round).  However, I get the sense that many people remain unconvinced.  Today, I’ll take a slightly different look at it to see if I can sway the remaining holdouts.  I apologize for the repetition in subject-matter, but this is an EXTREMELY important topic, as it goes to the heart of Optimal Draft Strategy.

First, I want you to ask yourselves if you agree with the following statement:

GMs should, in general, take the Best Player Available (“BPA”) with every draft pick.

I’m guessing most readers here would back that strategy, and I certainly do.  HOWEVER, it is not enough to just endorse that statement and use it as the basis for a draft strategy.  First, you must definite exactly what BPA means.

Here is where I am seeing some confusion and where I believe there is a big disagreement.  The people calling for taking Warmack high in the draft (forget about the Eagles for a moment, I’m speaking in broader terms), are using BPA as support.  The problem is that, in their BPA definition, they do not seem to be adjusting for positional value, which is a MASSIVE mistake.

Rather than attempting to explain why with a rational argument (for that see my last post), I’ll try to illustrate it.  Here are the top ten OTs and Gs from this past season, according to Pro Football Focus.  They are listed in order, with their original draft pick included.

Screen Shot 2013-03-04 at 11.34.40 AM

There are two big takeaways from this chart.  First, look at the draft picks.

I know I’ve said this repeatedly, but if you want an “elite” OT, you have to take him in the first round.  The same clearly does not hold for the Guard position.  This provides a clearer illustration of the opportunity cost argument I made last week.  There are really only two ways to get an elite OT in the NFL, draft him or sign him in FA. (I know we traded for Peters, but that’s a RARE exception).  Additionally, signing an elite OT in FA requires a huge contract.  Therefore, the BEST way to add a top OT to your team is to draft him in the first round.  Choosing a G means you are forgoing that opportunity.

The second point I want to highlight from the chart above gets at positional value.  Again, I want you to answer a question:

All other things being equal, would you rather trade Anthony Davis for Evan Mathis (or any other top Guard) or vice-versa?

I’m a big Mathis fan, but that’s a no-brainer.  Clearly most of the league agrees as well, hence the salary differences between OTs and Gs.

Assuming you agree, that means the #10 OT is worth more than the #1 G.

How about Brandon Albert versus Evan Mathis?

This one probably creates a bit more disagreement, but my guess is, on the whole, people would take Albert, who ranked as the 25th best OT according to Pro Football Focus.

Regardless, clearly there is a discrepancy in value between OTs and Gs.  Also, while I’ve focused on Gs and OTs, the same analysis can (and must be) done with all positions, and then incorporated into each team’s prospect rankings.  To not account for this is a huge mistake and one that appears is made by a lot of the writers who support taking a G high in the draft under a flawed concept of BPA.

To make sure I’m being clear: This should be incorporated into the “tiered” rankings I advocated previously.  So it is theoretically possible for a G to be the best choice with a top 15 pick, but it would require an amazing G prospect that can overcome the positional value difference to make it to the top 1 or 2 tiers, as well as every other player (from positions of greater value) within the same prospect tier being taken before said top 15 pick.

Hopefully this has convinced a few more of you, or at least provided a clearer explanation of what I was getting at last week.  This isn’t exactly Eagles-specific, because I don’t think there’s any way they take Warmack at #4, but it applies to any team with a top 15 pick.

Lastly, I’ll leave you with an analogy.  A frequent defense of G picks (and OL picks in general) in the first round has been that they are “safer”.  In general, that is true, as our strategy chart showed.  However, this is only one half of the equation.  As anyone in finance knows, evaluation is always a question of risk vs. REWARD.  To look at one side and ignore the other is a recipe for sub-optimal decision-making.

For example, given the choice between to raffles, and ignoring an external factors, which would you rather enter:

A) 60% chance of winning $100.

B) 75% chance of winning $75.

While the odds of success for raffle B are significantly higher, the correct answer (in a vacuum) is A, since the expected payout is greater.  It’s impossible to apply such specific values to players, but it’s a useful exercise nonetheless.  Here, imagine a DE as raffle A and an OL as raffle B.  To take the OL just because he offers less risk (25% chance of miss versus 40%) would clearly be a poor decision.

Advertisements

Hacking the Draft…Comprehensive Chart

Hopefully everyone has enjoyed the round-by-round breakdown.  While there are obviously a number of variables that can’t be controlled for, the pure statistical look at each position group has already provided some interesting insight.

Before I get to the big chart, let me just clarify exactly what I think this type of analysis is good for, then dig into an Eagles example.  Feel free to skip to the chart and come back.

The purpose of this is NOT to arrive at a set of rules by which teams should draft players.  Instead, it is meant to provide a general guideline, or a “default draft position”.  For example, yesterday I said that selecting DEs in the 3rd round is a terrible value proposition according to the data I’ve collected.  That doesn’t mean drafting a DE in the 3rd round is ALWAYS a terrible decision, it just means that for a team to make that decision, it must see or know something about the subject player that makes him CLEARLY much better than any other prospect available at that position.

The biggest flaw in NFL draft strategy, as far as I can tell, is each team’s confidence in its own ability to evaluate talent.  Regardless of the general manager, every team has historically had a very large margin of error when it comes to talent evaluation.  For instance, in the 1st round less than 6 in 10 LBs selected from 1999-2011 had or are having significantly productive careers (according to our definition.)

So what does that mean?  It means teams, in general, should be mostly focused on value during the draft, as opposed to parsing prospects.  This is perhaps never more apparent than when a team decides to trade up to select someone.  Let’s use the Eagles as an example.  NOTE: This is a very rough example, with numbers pulled from my ass, and is only meant to illustrate a larger point.

When the team traded up for Brandon Graham, it swapped 1st round picks with Denver and gave up two 3rd rounders as well (moving up from 24 to 13).

At the 13th pick, no DEs had been taken.  Between picks 13 and 24, 3 were taken, including Graham.  So that means the Eagles, in their analysis, decided that they had to take a DE (will not argue that decision here, though I was mad they passed on Early Thomas, and can produce witnesses that will verify I said that when it happened).

The only way the trade made sense was if the Eagles, in their DE analysis, decided that the odds of Graham becoming a stud DE were MUCH higher than the odds of JPP, Morgan, or Hughes becoming a stud DE.  Here is where the “margin of evaluation error” comes into play.

Using our historical draft data, we can calculate the odds of getting a starting DE with a 1st round pick and two 3rd round picks (I realize they were hoping more than an average starter, but stay with me).  Using the table below, we can calculate those odds to be 81.5%.  Using the Pro Bowl percentages from the earlier tables, we arrive at 37% for the odds of getting a Pro Bowl DE if you select DEs with a 1st round pick and 2 third round picks.  So here is the breakdown for “generic DEs”:

– 1st Round Pick – 24% chance of Pro Bowl, 67% chance of starting

– 1st Round Pick and 2 third round picks – 37% Pro Bowl, 81.5% starting.

Please note that this DOES NOT mean the Eagles made a bad decision.  Obviously the odds should be better for the 13th pick than for the 24th pick (we’ll get to that another time).  It DOES MEAN, however, that the Eagles, in their evaluation of Brandon Graham, should have been almost certain that he was more than 37% likely to be Pro-Bowl caliber, and more than 81.5% likely to start.

I have no doubt that they believed this, BUT, if they had applied a margin of error to their own analysis (as any good team should) prior to making the trade, they would have been unlikely to go through with it.  Let’s be extremely generous and assume the Eagles front office could peg these odds with a MOE of +/- 15% (WARNING: overly simplified statistics).   That means if they estimated Graham had a 90% chance of starting, his true odds of starting were almost definitely between 75-100%.

Immediately we can see an issue.  Even giving Graham incredibly high odds of starting (90%, which is more bullish than any team should be with any players outside the top few picks) and the Eagles a very generous MOE (+/-15%), the resulting range still does not exclude the 81.5% starting odds for the generic position of a first and two thirds (although it is near the bottom of that range).

That means the Eagles really should NOT have been (though I’m sure they were) confident that Graham would be better than just taking whichever DEs were available at #24 and in the third round, and hence, should not have made the trade.

I don’t mean to suggest that trading up is never a good idea, simply that the evaluative bar for whichever prospect is the target must be EXTREMELY high, and much higher than the standard currently being applied by most teams.

My final point:  Teams do not appear to take a probabilistic approach to drafting (which they almost certainly should), and I would guess that they do not actively overlay a margin of error onto their evaluations.  This is very much a “new-school vs. old-school” issue, similar to the statistical revolution in baseball, but IT IS NOT THE SAME.  It is largely a matter of GMs being willing to recognize and account for their own shortcomings and cognitive biases.  The franchises that can apply this will, in the long run, be more successful than those that don’t.  (Looking into which teams might be using this type of strategy will be another day’s post)

Ok, enough talk.  Here is the chart with every round included. I removed the All-Pro and Pro Bowl columns to make it easier to compare.  Dig into it and see what you think.  Later this week we’ll mine it for an “optimal default strategy”.

Screen Shot 2013-01-29 at 10.54.06 AM