After my last post about the historical odds of selecting an All-Pro caliber player in the top 5 versus 11-15, there were several comments about the potential effects of positional skew, so I will address those today.  For example, Quarterbacks are much more likely to be taken in the top 5 than anywhere else in the top 15 picks, so that could affect the overall All-Pro odds.

Before we get to some charts, let me note that one of the reasons I didn’t address this in the last post is that, when broken down by position, the sample sizes get very small.  Hence, we have to be careful not to jump to conclusions.  However, the skew effects by position are real and potentially very large, so it’s worth looking into.

I split the top of the draft into three segments, 1-5, 6-10, and 11-15.  Below are charts illustrating the positional breakdown for each.  We are looking at all drafts from 1995-2011.  The data labels are a bit ugly, but I wanted to show both the values and percentages by position.  Unfortunately, the colors do not match up by chart, so you’ll have to actually compare numbers if you want to dig in.

We can see pretty clearly that there is a big difference in positional distribution.  Before we re-evaluate the All-Pro odds, its worth noting:

– QBs, not surprisingly, are much more likely to get taken in the top 5.  The premium on the position means teams are much more competitive for them and therefore have to use more draft resources (i.e. higher picks) in order to get them.

– Wide receivers show a very strange pick distribution, with 9 selected in the top five, 18 going between picks 6-10, and just 6 selected from picks 11-15.  Haven’t yet come up with a reasonable hypothesis for this (mainly the drop-off from 18 to 6) but it may just be random.

So how does this effect our “elite” player odds?

Well it depends on the position, obviously.  The overall thesis from the last post was that teams are no more likely to find All-Pro caliber players in the top five than they are to find them with picks 11-15.  Overall, the data bears that out.  However, if you are looking for an All-Pro quarterback, its pretty clear that you need to take them in the top 5, if only because  top graded QBs don’t last very long (so most “1st Round” QBs go in the top 5).

What happens when we look at positions that don’t exhibit great frequency variation by pick?

First, let’s look at Defensive Ends.  In the above charts we can see that from 1995-2011, 11 DEs were taken in the top 5, 9 were selected from 6-10, and 14 went between picks 11 and 15.  Were the top 5 picks any more likely to become elite players?

Not really according to the data, but since the samples are small, we should probably stick to saying it’s “unclear”.  In any case, this chart certainly supports the notion that it’s probably better to trade down from the top 5 (at least if the team is set on choosing a DE).

Now let’s look at OTs, another position for which there is relative consistency in terms of frequency by draft pick.

Again it’s unclear.  Here the odds in the top 5 are clearly better than from 11-15, but with just 8 picks in the latter range, we’re really taking a leap to infer anything significant.  Also, the odds here are better between 6-10 than from 1-5.

Outside of QBs, which are so skewed by frequency that we can’t do much with the data, the only position that appears to show a significant decline in odds of success as you move later in the top 15 is Wide Receiver, and the numbers here are shocking.

To make it a little more interesting, here are the 6 WRs that were selected between 11 and 15:  Yatil Green, Troy Edwards, Rod Gardner, Donte Stallworth, Lee Evans, Michael Clayton.

I’m guessing most people here have only heard of 3-4 of them…

I could go position by position and give you the All-Pro chart for each, but I don’t think it’d be that valuable.  Some positions show slight odds declines, but remember, the question isn’t just “Are the odds better with a top 5 pick?”, it’s “is the difference in odds great enough to outweigh the compensation gained by trading down?”

In general, my previous point stands:  Teams would be better off trading down from the top 5 (while remaining in the top 15).  This doesn’t hold if you are taking a QB, but for everyone else, nothing I’ve seen contradicts this strategy.

But how could this be?  I admit it doesn’t make a lot of intuitive sense.  All of the “elite” players chosen between 11-15 were available with picks 1-5, so shouldn’t the odds have been better there? They should, but they’re not, mainly because NFL scouting isn’t perfect.

Overall, NFL scouts appear to be very good at segmenting players by general ability (tiers if you will).  That’s why elite players are predominantly selected in the first round and almost never slip past the second round.  HOWEVER, scouts do not appear to be good at sub-sorting those tiers.  So they can pick out the handful of DEs (or whatever position you want) that are the best, but ranking the players within that group is extremely difficult.  This is where the problem of false precision comes into play.  If teams would recognize that limitation, they would consistently trade down, selecting the last available player in the best “tier”.  GMs and scouts, though, appear to believe they can reasonably predict which individual player is going to be better,  leading them to use draft resources inefficiently.

If you play fantasy football, you should be familiar with this.  I’m betting at least some of you sort players by tier prior to each season, and draft accordingly.  So you don’t get hung up on which QB is the “best”, you just make sure you get one from the best group, understanding that it’s impossible to predict exactly who among that group will have the most successful season.

I’m going to do a deeper analysis of this concept soon and look at the relative performance of players according to order drafted (for instance, does the 1st player taken at any position outperform the next player taken at the same position), but for now, the Star/Joeckel/trade-down strategy remains (assuming those players are as highly regarded as currently indicated, Joeckel seems to be losing some ground to other OTs).