Who are the Riskiest/Safest Top Prospects?

If you haven’t seen the new Total Prospect Rankings, see either yesterday’s post or the PVM Rankings tab at the top.

There are a number of ways we can slice and dice this information, and over the next couple weeks I’ll look at a bunch of them.  Today, we’ll look at the “riskiest” prospects this year.  Note we are only looking at the top subsection of players (137 total, of which I posted 130), so we aren’t including players who are likely to be late round picks.

As I discussed yesterday, “risk” in the draft is composed of two components, the individual player’s risk and the larger positional risk, as quantified by the historic success of players drafted at each position.

First let’s look at the individual component.  Here is a chart showing the prospects with the highest standard deviations in their consensus ratings.  Note that the measure here is HALF the true standard deviation.  Also, for this chart I’ve only used Top 60 prospects.  The worse prospects tend to have higher standard deviations, so using the whole sample results in a bunch of guys that aren’t really worth talking about just yet.

The columns should be self-explanatory, but: “Consensus” is the average scouting score.  “.5 SD” is the measure we are looking at.  “Pos Miss Rate” is the historical failure rate for each position.  “Risk Adj” is the combined risk component, as described yesterday.

Screen Shot 2013-04-10 at 12.10.48 PM

Immediately, the two QBs at the top jump out.  EJ Manuel and Ryan Nassib, both potential targets for the Eagles in round two (or late round 1) carry very high “individual risk rates”.  The three scouting groups I’ve used have widely different opinions of these players.  Therefore, they are riskier choices.

Other notable inclusions on this list are:

– Sylvester Williams, DT.  Ranked 22nd overall by the Consensus Scores, he drops to 33 in the TPR system (due in part to his placement above).  He carries the 6th highest individual uncertainty score of all the top prospects.

– John Cyprien, S.  He is a popular target for the Eagles 2nd round pick, but does not fair well in the TPR system.  There is relatively wide disagreement among scouts about his ability, and while he ranks 30th in the Consensus Scores, he drops to 43rd in the TPR rankings.  Note that the Eagles 2nd round pick is nearly halfway between those two rankings, meaning he remains a viable option for the Eagles at pick #35.

– Towards the bottom, we can see Sharrif Floyd carries the highest individual uncertainty among Top 10 prospects.  This is part of the reason I am firmly in the Star>Sharrif camp.

Now here is the chart showing the players with the LOWEST individual risk grades:Screen Shot 2013-04-10 at 12.24.41 PM

This is perhaps the more interesting chart for Eagles fans, as it includes many of the potential targets for the 4th overall pick.

– Chance Warmack remains a pain in the ass.  I originally conceived of the positional adjustments in an effort to explain why Chance is a terrible use of resources in the Top 10. However, through two iterations he remains near the top of the TPR draft board.

Here I have to stress two things about Warmack that people aren’t mentioning enough:

1) He is the highest rated prospect by Consensus Score.

2) He has the lowest standard deviation, suggesting there is widespread agreement that he is the best prospect.

I’m close to the point of saying that our (and MY) preconceived notions about positional value may be blinding us to what should be an obvious assertion, namely: Chance Warmack should be a top pick.

I’m not there yet, but the entire idea here was to take subjectivity and bias out of the equation, and on that basis, Chance Warmack looks pretty good.

– Dee Milliner, Dion Jordan, and Star Lotulelei all place near the top of the board.  That means, on an individual basis, we should be more confident in their scouting grades then in the grades of players lower on the list.

How about Total Risk?

Now let’s look at the total risk component.  As described yesterday, the total risk component is weighted heavily to Positional Evaluative Error, with individual uncertainty included, but with less impact.

This is one area where I am not 100% comfortable with the model.  Should individual uncertainty carry more weight?  I don’t know, but maybe, and it’s something I’m thinking about as we get closer to the draft.

Anyway, here is a chart showing the Top 60 prospects with the HIGHEST risk components:Screen Shot 2013-04-10 at 12.34.53 PM

Nobody should be surprised to see EJ Manuel and Ryan Nassib at the top of the list.  A lot of individual uncertainty + a large positional risk rate = a big risk component.  These guys may become good/great players, but given where they will likely be drafted, they represent the riskiest picks in the entire draft.

Others of note:

– We already discussed Williams, Cyprien, Floyd.  For obvious reasons, they are included above.

– Keenan Allen is another one to note. Ranked 16th overall by Consensus Score, he is one of the “riskiest” top prospects, despite the fact that the TPR system only drops him to 19th overall.

While the guys above probably don’t seem familiar to most of you, they are all players that we’ll be seeing/talking about towards the bottom of the 1st round and through the 2nd round of the draft.  Might as well start familiarizing yourself with them now.

Now for the LEAST RISKY top prospects:Screen Shot 2013-04-10 at 12.41.59 PM

Not surprisingly, we see mostly offensive linemen at the top of the list.  The biggest name for Eagles fans is Eric Fisher, who is probably the most likely pick for the Eagles at #4.  Based on the current measure of risk that I’m using, Fisher is the 3rd SAFEST top prospect.

Some other things to keep in mind:

– Pay close attention to the CBs listed near the top of the list.  Guys like Rhodes and Taylor could be great pick-ups for the Eagles if they fall to the second round.

– Menelik Watson is another guy to keep an eye on.  It’s possible (though I think unlikely) that he’s available at #35 and, should the Eagles pass on Fisher at #4, would be a prime target for the team.

As I said, I’ll be looking at this info in a number of different ways, but please feel free to suggest anything in particular that you’d really like to see, either through twitter, email, or the comments.



8 thoughts on “Who are the Riskiest/Safest Top Prospects?

  1. I’m really enjoying all of this analysis. Keep up the good work! One thing that isn’t covered in the risk part of your analysis is health (Jones and star come to mind). I realize it may be beyond the scope of what you are trying to accomplish (and very hard to quantify) but it might help with a better overall picture. Also, I think part of the reasoning behind not drafting a guard at the top of the draft is impact. In the top 15 picks your looking for an impact player. Not just a pro bowler, but someone to build your team around. You build your line around tackles, not guards. You build your offense around QBs, not WRs. Etc. Again I realize that is hard to quantify and may depend on team (scheme places different importance on different positions and different holes create different levels of impact, etc.)

    • Health is definitely an issue that I can’t yet quantify. However, durability factors into the scouting ratings, so there might be some mitigation there.

      As far as the impact goes, thats what the positional modifiers are for, so that’s already accounted for in the system. If you look at the rates by position, Guards are at the very bottom, so Warmack loses a lot of ground there, but remains at the top of the list.

      Just a case of how the draft board looks this year. I think in the future we will very rarely see a Guard rank this high in the TPR system.

      On Wed, Apr 10, 2013 at 2:59 PM, Eagles Rewind

  2. Love your analyses, keep up the great work!

    My only bone to pick with this post is how “safe” Dion Jordan is portrayed. His rankings by ‘experts’ are almost all based around his combine workout, physical attributes, and potential. His collegiate record and gametape are not good. IMO that makes him a huge risk that wont show up in these stats.

    • Very true. The system is based heavily on the idea that scouts, in general, do a very good job (which appears to be true). However, if they screw up (by overweighting the workouts) it will show up here.

      However, this system isn’t meant to be perfect (not that it ever could be), it’s only meant to be BETTER THAN the individual rankings of any scout. Looking at it from that perspective, the system should definitely outperform, since any individual scout’s overreactions will hopefully be counterbalanced by the other ratings. If every scout screws up, then it’ll hurt, but obviously it will be just as damaging to those scouts’ rankings.

      In the future, it might be possible to quantify something like college production and work that into the formula, but for now I’d like to keep it simple (relatively) and see how we do.

      On Wed, Apr 10, 2013 at 4:43 PM, Eagles Rewind

  3. First off, I love the site, I don’t know if other teams have someone taking this sort of analytical view but I feel lucky to be a fan of a team for which this analysis is available.

    On your rankings – I wonder if it would be worth tracking players over time to see how your approach holds up and how accurate the various ratings turn out to be, including your own. Obviously, you won’t have much data for years, but it would be a good thing to be able to look back on. If you’re able to spell out in advance your future approach to valuing players, that would be even stronger.

    Generally, when doing this sort of analysis, I think it really gains when you can set out how you will gather feedback which will suggest what you’ve been doing right and what needs to be improved, preferably by making some sort of prediction which can be checked in future.

    Thanks for all your work, looking forward to reading more!

    • Thanks for the feedback, glad you enjoy. To answer your question, I will DEFINITELY be tracking the success of each ranking/rating system. As you’ve said, I dont have a lot of data going backwards, so I can’t do a lot of “backtesting”. However, that may be a good thing in the long run, as it means I won’t fall victim to data-fitting.

      Take a look at the PVM rankings tab on the site. That’s my biggest attempt yet to establish an objective, testable system.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s