Patrick Causey, on Twitter @pcausey3
Many of you are probably wondering why the hell I am writing about the draft on the week leading up to the start of the regular season. Fair point. But in my last post, I said Howie Roseman had a mixed track record in the draft. But I largely assumed that was the case based on my understanding of the Eagles draft history.
I started to fixate on whether I had any concrete data to back it up. My fixation led to research, which led to a 2,000 word article on the draft. So here we are.
I’m going to update this post during draft season, but my goal is to quantify how successful Roseman has been in the draft compared to his peers. The first thing I learned? Evaluating the evaluators is as much of a crapshoot as the NFL draft process itself, a sentiment echoed by FiveThirtyEight.com this past April.
But there is enough evidence out there to evaluate Roseman’s track record to some extent. Here, I relied on three sets of data during the time period in which Roseman was the general manager (2010 through 2014):
- (1) comparing the percentage of games that Roseman’s picks started with the historical expected start rate for a player drafted in the same round;
- (2) comparing Roseman’s rate at drafting Pro Bowl and All Pro players to the league wide average and compared to some of the best in the game; and
- (3) comparing the approximate value of the players that Roseman selected with other players from the same draft class.
So what did I find? The overall picture is a bit murky. Some metrics suggest Roseman is a below average talent evaluator compared to his peers, while others suggest he is above average. Where there is no disagreement, however, is Roseman’s rate at identifying top tier talent — it’s not good, and explains why the Eagles haven’t seriously competed for a Super Bowl in the last six season.
Before getting to the data, a quick note. Lurie’s “shared responsibility” front office structure makes it impossible to identify who had final say over draft picks. Was it Reid? Roseman? Kelly? I realize that everyone has their own opinion on this. But I am not concerned with parsing that out here. I am only concerned with how the Eagles have drafted under Roseman (hence the title); I’ll defer to others to assign credit or blame for a particular pick.
Judging by “Start Rate”
There are a number of ways by which you can measure the success rate of general managers, each of which carry their own inherent flaws. Caveats aside, I am evaluating the relative success of our draft picks by comparing the percentage of games each Eagles draftee has started to the expected start percentage based on data I compiled from Pro-Football-Reference.com.
To determine the expected start rate, I examined every draft pick taken from 2010-2014 and calculated the average start rate for each round of the draft during that time period. So, for example, I found that first round draft picks from 2010-2014 have started, on average, 63.62% of their possible games. As you will see in a minute, Marcus Smith (started 0% of his possible games) would be considered a failed pick, while Fletcher Cox (89%), would be considered a success.
You can access the entire data set I compiled here, which provides some useful information regarding not only the overall strength of a draft class, but also the strength of a particular round within each draft. But this chart summarizes my overall findings:
| Round | Start % | Range |
| 1 | 63.6% | 58.4% – 65.6% |
| 2 | 42.5% | 39.5% – 45.8% |
| 3 | 34.8% | 29.4% – 40.4% |
| 4 | 23.4% | 18.7% – 29.7% |
| 5 | 17.5% | 15.6% – 20.7% |
| 6 | 12.3% | 10.7% – 13.8% |
| 7 | 10.6% | 7.8% – 14.8% |
Not surprisingly, the percentages track what we would have expected: higher round draft picks start more frequently than the lower round picks. I did not see any trends in the data that required me to adjust the raw numbers. For example, with the league average career at 3.3 years, one might expect higher start percentages for more recent drafts. That was not the case. So I am sticking with these overall numbers.
Start Rates of Roseman’s Picks
Here is each draft pick made by the Eagles under Roseman from 2010 until 2014. I am excluding this past year for obvious reasons and 2015 since Roseman was not involved in personnel decisions. Numbers highlighted in red indicate a player falling short of his expected start rate, while numbers in green show a player that exceeded it.
2014
| Round | Player | Position | Start % | Expected Start % |
| 1 | Marcus Smith | LB | 0% | 63.6% |
| 2 | Jordan Matthews | WR | 71.8% | 42.5% |
| 3 | Josh Huff | WR | 12% | 34.8% |
| 4 | Jaylen Watkins | CB | 0% | 23.4% |
| 5 | Taylor Hart | DE | 3% | 17.5% |
| 6 | Ed Reynolds | DB | 9% | 12.3% |
| 7 | Beau Allen | DT | 6.25% | 10.6% |
2013
| Round | Player | Position | Start % | Expected Start % |
| 1 | Lane Johnson | T | 91.6% | 63.6% |
| 2 | Zach Ertz | TE | 29.7% | 42.5% |
| 3 | Bennie Logan | DT | 79.1% | 34.8% |
| 4 | Matt Barkley | QB | 0% | 23.4% |
| 5 | Earl Wolff | DB | 14% | 17.5% |
| 7 | Joe Kruger | DE | 0% | 10.6% |
| 7 | Jordan Poyer | DB | 5% | 10.6% |
| 7 | David King | DE | 0% | 10.6% |
2012
| Round | Player | Position | Start % | Expected Start % |
| 1 | Fletcher Cox | DT | 89% | 63.6% |
| 2 | Mychal Kendricks | LB | 82.8% | 42.5% |
| 2 | Vinny Curry | DE | 0% | 42.5% |
| 3 | Nick Foles | QB | 60.9% | 34.8% |
| 4 | Brandon Boykin | DB | 10.9% | 23.4% |
| 5 | Dennis Kelly | T | 23.4% | 17.5% |
| 6 | Marvin McNutt | WR | 7.8% | 12.3% |
| 6 | Brandon Washington | G | 0% | 12.3% |
| 7 | Bryce Brown | RB | 10.9% | 10.6% |
2011
| Round | Player | Position | Start % | Expected Start % |
| 1 | Danny Watkins | G | 22.5% | 63.6% |
| 2 | Jaiquawn Jarrett | S | 11.2% | 42.5% |
| 3 | Curtis Marsh | CB | 0% | 34.8% |
| 4 | Casey Matthews | LB | 20% | 23.4% |
| 4 | Alex Henery | K | 62.5% | 23.4% |
| 5 | Dion Lewis | RB | 7.5% | 17.5% |
| 5 | Julian Vandervelde | G | 0% | 17.5% |
| 6 | Jason Kelce | C | 77.5% | 12.3% |
| 6 | Brian Rolle | LB | 16.25% | 12.3% |
| 7 | Greg Lloyd | LB | 0% | 10.6% |
| 7 | Stanley Havili | RB | 12.5% | 10.6% |
2010
| Round | Player | Position | Start % | Expected Start % |
| 1 | Brandon Graham | DE | 23.9% | 63.6% |
| 2 | Nate Allen | S | 75% | 42.5% |
| 3 | Daniel Te’o-Nesheim | DE | 28.1% | 34.8% |
| 4 | Trevard Lindley | DB | 1% | 23.4% |
| 4 | Keenan Clayton | LB | 1% | 23.4% |
| 4 | Mike Kafka | QB | 0% | 23.4% |
| 4 | Clay Harbor | TE | 36.4% | 23.4% |
| 5 | Ricky Sapp | DE | 0% | 17.5% |
| 5 | Riley Cooper | WR | 56.25% | 17.5% |
| 6 | Charles Scott | RB | 0% | 12.3% |
| 7 | Jamar Chaney | LB | 23.9% | 10.6% |
| 7 | Jeff Owens | DT | 0% | 10.6% |
| 7 | Kurt Coleman | DB | 48.9% | 10.6% |
So what do these numbers tell us?
- The Eagles had 48 total picks from 2010-2014.
- Under Roseman, the Eagles met or exceeded the expected start rate for 17 of the 47 picks.
- So the Eagles “success rate” in the draft under Roseman is 36.17%.
- Put another way, 30/47 draft picks failed to meet their expected average start rate.
We can probably take issue with how certain players are characterized. For example, this approach considers Brandon Graham, Zach Ertz and Vinny Curry as misses because they didn’t start the requisite number of games. That’s clearly not right. But it also marks Alex Henery, Nate Allen and Brian Rolle as hits, and those characterizations seem equally suspect. In other words, I think it’s reasonable to assume that this evens out in the end, but I won’t argue if you make some changes to the classifications.
At first blush, these numbers suggest that Roseman/the Eagles are falling short of what we would expect. When you are meeting “average” production only 36% of the time, that seems like a problem. But I would feel more confident in that conclusion if I ran this calculation for all 32 general managers during that five year window. That would give me a much more accurate benchmark by which to evaluate how Roseman has done. I am going to leave that for leading up to the draft, since it is going to take a herculean effort to run those calculations. It will be much easier for me to spread out running the numbers over the course of the season given my other time commitments.
I filled in the gaps by looking at other research done on success rates of draft picks. The best data set I could find came straight from the horses mouth: ESPN.com’s Paul Kuharsky interviewed several general managers that provided their teams’ internal statistics regarding draft success rates. Teams have access to much more comprehensive data sets than I do, so I feel confident relying on this.
Here is one general manager’s bench mark for success: 1st and 2nd round picks: 56%, third round picks 35%. Those numbers are almost identical to the numbers I have compiled (combined 53.05% for 1st and 2nd rounders, 35% for third). So I feel comfortable using this as a baseline for evaluating Roseman’s picks, even if the relative definition of “success” might be different.
The Eagles had 11 first and second round picks from 2010-2014. Based on the data above, five of 11 were hits, or 45%. An argument can be made to count Graham, Curry, and Ertz as hits, but Allen as a miss. Under that assumption, Roseman’s hit rate is a much more impressive 63%.
The Eagles also had 5 third round picks during this time period, and hit on 2 of them, or 40%. I wouldn’t change any of the hits or misses as they are defined above, so Roseman beats that rate.
Finally, one general manager suggested that teams aim to gain 2.3 starters out of every 7 picks (Bill Polian was much more aggressive, saying good teams hit on 4.5 out of 7 picks, but he included undrafted free agents in that calculation, something I have not accounted for in this study). Using that 2.3 mark, we should expect 15.36 starters out of the 48 picks the Eagles had. Based on my evaluation, the Eagles drafted 10 such players: Cox, Kendricks, Johnson, Ertz, Logan, Matthews, Coleman, Graham, Curry and Kelce. Maybe we could include Allen and Henery in the mix. But getting to 15 would require us to consider players like Lewis, Foles and Boykin as starters. And I don’t think the evidence we have to date supports that conclusion. So Roseman falls short of this mark.
Bottom line: Roseman graded out positively under some metrics, but came up short on others. I think we will get more clarification once I run the numbers for other GMs, so for now let’s split the baby and call his rate middle of the pack.
All Pro and Pro Bowl Players
Another way to measure Roseman’s draft acumen is to compare the rate at which he has drafted Pro Bowlers and All Pros with the rest of the league.
According to Pro-Football-Reference.com, there have been 1,272 players drafted from 2010-2014. Out of those draft picks, 116 have been named All Pros, which is 9% of the players drafted. That means we should expect each team to draft, on average, 3.625 All Pros during that time period.
The Eagles have zero. (Although I firmly believe that Fletcher Cox should have received an All Pro award, but I digress).
What about Pro Bowls? Again courtesy of Pro-Football-Reference.com, there were 207 Pro Bowl selections for players drafted from 2010 to 2014. That is an average of 6.46 Pro Bowlers per team.
The Eagles have three: Kelce (2014), Cox (2015), Nick Foles (2013). Even if we count Kurt Coleman’s Pro Bowl with the Panthers last year (and I don’t think we should), Roseman still falls short of the league wide average.
A recent study on NFL.com confirms Roseman’s place below the elite general managers when it comes to identifying and drafting elite talent. Mike Huguenin found the top 10 general managers had the following success rate at drafting Pro Bowl players:
- Ryan Grigson, Colts: 9.09%
- Ozzie Newsome, Ravens: 9.09%
- Mike Brown, Bengals: 9.52%
- Rick Spielman, Vikings: 10.34%
- Ted Thompson, Packers: 10.42%
- Bill Belichick, Patriots: 10.53%
- Kevin Colbert, Steelers: 12.3%
- Rick Smith, Texans: 12.33%
- John Schneider, Seahawks: 14.58%
- Mickey Loomis, Saints: 14.6%
- Jerry Jones, Cowboys: 14.72% (What the hell?!)
Under Roseman, the Eagles success rate is 6.25%.
Some might question how fair this study is given that the Eagles have drafted towards the tail end of the first round more often than not. It’s a reasonable question, but one that doesn’t hold up under closer scrutiny. Look at that list again: it includes the Patriots, Packers, Steelers, Vikings, Seahawks, Saints, and Ravens, to name a few, teams that are routinely drafting at the end of the first round.
That Roseman struggles at drafting elite players is not controversial. If you were asked to identify the biggest weakness of the Eagles, outside of lacking a franchise quarterback, you would likely say that they lack elite playmakers. This lends credence to that idea.
Evaluating by Approximate Value
The final way I judged Roseman’s drafting ability is by using Pro-Football-Reference.com’s approximate value metric. Specifically, I examined where the Eagles draft picks ranked in terms of approximate value compared to their peers. This approach confirmed the results above: the Eagles do a decent job at accumulating league average players, but fall short when it comes to getting elite ones.
Consider this, of the 48 players the Eagles drafted, only one — Fletcher Cox — ranks in the top 10 in approximate value for his respective draft class. That’s a poor rate compared to the top franchises in the league.
Or consider this: in two consecutive drafts, 2010 and 2011, the Eagles did not get a single player that currently ranks in the top 32 of approximate value (i.e., a first round talent).
And finally, the Eagles drafted five players that rank in their respective top 32 (Cox, Johnson, Logan, Kendricks and J Matt). But that pales in comparison to the Seahawks (11), Bengals (9), and Steelers (8).
Here is a breakdown of the analysis:
2010:
- 255 players drafted.
- The Eagles do not have any players rated in the top 32 of approximate value (i.e., a 1st round pick).
- For comparisons sake, the Seahawks have 4, the Broncos and Steelers have 3, and the 49ers and Patriots have 2.
- The highest rated player is Nate Allen, who ranked 50th, with an approximate value of 26.
- Other players in the top 100: Kurt Coleman (65), Riley Cooper (75), Brandon Graham (93), Jamar Chaney (99).
2011:
- 254 players drafted.
- The Eagles again do not have any players rated in the top 32 of approximate value.
- The highest rated player, Jason Kelce, just missed out, ranking 33rd overall.
- The Seahawks and Bengals have 3, while the Cowboys and Broncos have 2.
- Confirming our belief that the 2011 draft was historically bad, the Eagles have only one other player in the top 100, Casey Matthews, who ranked 99th in career approximate value.
2012:
- 253 players drafted.
- The Eagles have two players in the top 32: Fletcher Cox (6th) and Mychal Kendricks (31). Nick Foles is just outside the first round grade, ranking 34th overall. Brandon Boykin snuck into the top 100, as the 100th overall ranked player.
- The Seahawks again have the most with 4, while the Patriots, Panthers, Buccaneers, Colts, Dolphins and Redskins all have 2.
2013:
- 254 players drafted.
- The Eagles have two players in the top 32: Lane Johnson (17th) and Bennie Logan (26th). Zach Ertz is just outside the top 32, ranking 38th.
- Probably the most notable thing about this draft is that the Seahawks didn’t have any player in the top 32 after crushing it the last three seasons. Other teams that did well in this draft include the Packers, Chargers and Bears, who have 2 players in the top 32.
2014:
- 256 players drafted.
- The Eagles have one player in the top 32, Jordan Matthews (26). They do not have any other players in the top 100 of this draft.
- The Jaguars had 3 in the top 32, while the Dolphins Raiders Bills and Vikings had 2 each.
The biggest concern is that the Eagles have only one player in the top 10 of their respective draft classes. This falls short of some of the best franchises in the league. Perhaps Logan, Ertz and/or Johnson make the jump over the next year or two. Or maybe Wentz becomes that guy. Either way, the Eagles are going to need to get more elite talent if they are going to realistically compete for a Super Bowl.
Final study: Rotoworld evaluated the 2011-2015 drafts based on approximate value. It doesn’t perfectly track our timeline, but it’s pretty damn close. As you can seee in the graph below, they track the combined approximate value of each franchise’s draft picks compared to the number of draft picks used.
As you could have guessed, Seattle blows everyone out of the water given their late round hits with Russell Wilson, Richard Sherman, Kam Chancellor, etc. Houston, Cincinnati and Carolina are also very efficient in terms of maximizing value for the number of picks they used. While the Giants, Saints, Lions and 49ers have come up short on approximate value.
Philadelphia is averaging about dead center in the NFL in terms of number of picks and approximate value.
The Take-Away
We can confidently say that under Roseman, the Eagles have come up short identifying and drafting elite talent in the draft. There is less certainty regarding how they have done overall. The evidence suggests that they are right around league average, but I’ll feel more confident once I can dig into the success rates of every general manager in the league.
What do you think? Leave a comment below. I am interested to hear your thoughts and see if you have any ideas on additional ways to examine this.
Eagles Rewind 