Continuing our draft research, I attempted to create a method for determining which teams were the best and worst at drafting over Andy Reid’s tenure, with 2012 excluded since it’s too early to judge most of this year’s rookies.
Before the results, I have to explain how I got there (feel free to skip if you don’t care about the methodology):
First, in order to evaluate players, I used the Career Approximate Value statistic from Pro-Football-Reference.com. This is FAR from a perfect measure, but I don’t know of any better way to compare player production for different positions.
Second, Career AV is a cumulative measure. So players with longer careers will tend to have higher Career AVs. To compensate for this, I divided each players CAV by the number of seasons they were in the NFL. It’s important to note that this skews the data in favor of players who were productive over long periods of time versus those (i.e. Shawn Andrews) who were spectacular for a short time and non-existent the rest.
I then totaled those measures for each team, giving us a rough idea of the amount of production derived from each team’s drafts.
However, we aren’t done yet. In order to adjust for the differing value of each draft pick, I calculated the average point value of each round (according to basic NFL draft chart), then used those measures to calculate the total “Draft Points” used by each team. That way, teams that use a lot of 1st round picks to find production will not be equal to teams that found the same production with lesser picks.
Finally, I divided the Total Adjusted CAV for each team by the total Draft Points used to obtain that production.
The results? Some expected, some shocking. Here are the ten teams (in order) that ranked best from 1999-2011 in identifying and selecting productive players in the draft:
Tennessee, Chicago, Indy, Dallas, Green Bay, Atlanta, Carolina, San Diego, Philadelphia (9th), Houston.
Here are the ten worst (ordered worst to best):
Detroit, St. Louis, Oakland, San Fran, Washington, Minnesota, Kansas City, Cincinnati, Buffalo, Tampa Bay.
Note: This is an attempt to rank teams based on the efficiency with which each used their draft picks.
For the skeptics: A first glance at the best and worst drafting teams would lead one to believe there is a flaw in the methodology. The Titans? Really? On a hunch, I graphed each team’s Adjusted AV/Draft Point (the measure I walked through above) against each team’s wins over the subject time period (Texans not included since they were not in the league for the whole time period). Here is the chart:
Interesting, right? We see a clear positive correlation between the drafting measure we calculated and team performance (wins). Not only that, but it’s a pretty strong relationship (coefficient of .55).
This doesn’t prove the effectiveness of my methodology, but it is certainly what we would expect to see given an accurate measure of drafting skill. Teams that draft better should, on average, perform better on the field.
Here is a graphic showing each team’s Adjusted AV per Draft Point, with the Eagles highlighted in red.
So what did we learn? The Eagles actually have been pretty good at using draft picks since 1999 (and Detroit has been absolutely terrible). Keep in mind that this just measures a team’s ability to find productive players with each pick. It doesn’t account for whether picks achieved success while with the team that drafted them. For instance, the Eagles get credit for Derrick Burgess, even though most of his success came with another team.
Also remember that this is per pick, not total production (i.e. New England had the 4th highest total Adjusted AV, but also used the second most picks to get there).
I’ll be digging further into the data to see what else we can get out of it, but I believe we can drop the whole “Andy Reid sucked at drafting” meme. It’s also a good reminder that as much as some fans hated Reid, over his tenure here, most other franchises were much worse.
Very interesting, although it begs the question…how were the Titans/Bears/etc. so high? Were their scores driven by huge superstars or a couple hidden gems vs. very consistent performance?
I also would’ve assumed Pittsburgh would’ve fallen in the top 10 as they’ve got a big reputation for drafting well.
I would also argue that the analysis is a little limited as teams get credit for performance of their picks even after they leave. Obviously it’s tough to break that out or you would’ve done it, but I wonder if that has a significant impact on the metric.
The idea was to solely measure each team’s ability to pick good players. So the Eagles should get credit for identifying Burgess even though he wasn’t great until he left. They still identified his talent.
I plan on doing a post that breaks out points for your comment about superstars and hidden gems.
For instance, I haven’t done any calculations, but it looks like the Titans are really good at avoiding complete busts (zeroes by the AV metric).
The titans being first is certainly surprising, but them being in the top ten makes sense I think. They were good under Jeff Fisher for many years, especially at the beginning of Andy’s time in Philly. Since he had such a long tenure it’s easy to forget the way things were when he started. I also think the titans have traditionally struggled to keep veterans around once they become free agents. I’ve got no data to back up those claims however. Keep up the great work, I love reading this blog