Stuff We Care About


 

 

Search
Register
This form does not yet contain any fields.
    Click Image to Purchase

    Book Reviews
    The Authors
    « Mike Leach Debuts on Reddit! | Main | R.I.P. Adam Yauch »
    Saturday
    May052012

    Why College Recruiting Services Don't Work

    Last week the  76th edition of the NFL draft ended when the Indianapolis Colts selected Northern Illinois Huskies quarterback Chandler Harnish with the 253rd and final pick of the 2012 draft.

    Harnish was a lightly recruited dual-threat quarterback from Wells County, Indiana (pop. 27,636) who received offers from just two schools, Ball State and Northern Illinois.

    Harnish would go on to have a storied career at Northern Illinois where he set 30 passing, rushing and total offense records in four years as the Huskies’ starting quarterback. Harnish was also recognized as a three-time Academic All-MAC selection and was a first team All-MAC choice in each of his last two seasons.

    In hindsight, it seems that the recruiting services somehow overlooked this unknown future star from rural Indiana.

    In 2006, Chandler was among 160 two-star rated high school senior quarterbacks. That year, 72 quarterbacks received a three-star or higher designation, including current NFL starters Sam Bradford, Matt Stafford, Josh Freeman, and Tim Tebow (Bradford and Tebow were also Heisman trophy winners).

    NFL Draft Outcomes for 2006 College QB Prospects

    Among these 72 higher rated quarterbacks, just nine would go one to be drafted in the NFL.

    Out of four 5-star rated quarterbacks in the class, two players (Stafford and Tebow) were drafted.

    Of the eleven 4-star quarterbacks, only Josh Freeman and Jake Locker were drafted; 18.2% of the 4-star class.

    Among the 57 three-star quarterbacks, just five players were drafted; 9.6% of the three-star class.

    Harnish and North Carolina’s T.J. Yates were the only two quarterbacks drafted among 160 two-star prospects.

    These results are not just isolated to the incoming quarterback class of 2006. Instead, as we detail below, these outcomes are typical of the recruiting services’ general inability to predict future NFL or even college success.

    Sample Universe

    Rivals.com has been evaluating high football players since 2002. From 2002-2008, Rivals.com compiled a database of 40,181 players including an elite group of 200 high school players upon whom the recruiting service bestowed its most coveted five-star rating.

    Based on its player evaluation and ranking system, Rivals.com considered these players to be among the top 99.5% high school players in its database, who, in almost all cases, the service also ranked as the country’s number one high school player by position.

    These ratios understate just how unique the college recruiting services deem these players to be.

    As Braden Gall of Athlon Sport reminds:

    Those are just the ones that get evaluated and receive the subsequent star ratings, however.

    According to MaxPreps.com, there are roughly 15,000 high school football teams in this country. That is approximately 300,000 senior football players in any given year (15,000 teams X 20 seniors per team). Those aforementioned percentages become microscopic when applied to the true player pool.

    One-thousandth of one percent of high school senior football players will ever receive a five star rating. Keep that in mind.

    Based on the manner in which these grades are awarded, it stands to reason that Rivals.com believed these players had the highest probability of being successful at the college level, and by extension the players most likely to be drafted into the pros. 

    Assessing the Recruiting Services

    How did these five-star prospects - the players whom the recruiting services considered the best of the very best players in the country - fare in the NFL draft?

    In a word, poorly.

    Over a seven year period, among the 200 prospects who received a five-star rating, just 30 were selected in the NFL draft’s first round.

    Draft Outcomes for Five-Star College Football Prospects (2002-2008)

    A more generous assessment  shows that overall, from 2002-2008, the NFL drafted 53.5 percent of former five-star prospects over all seven rounds.

    These figures demonstrate that, at best, when a college recruiting service assigns its highest possible five-star recruiting rating to a prospect, the player has a slightly better chance of being drafted than a coin toss.

    This is about as good as the recruiting services gets.

    After that, it’s a crapshoot.

    The Four-star Abyss

    The likelihood of four-star, three-star and two-star players being drafted falls to such an extent, we wonder why they are rated at all.

    First Round NFL Draft Odds by Recruiting Grade

    Jared Diamond explains that the low ratios are a partially a product of the denominator effect:

    [T]his dynamic is a function of how large a pool the three-star-and-below players are. There are typically only around 30 five-star prospects per year and 300 to 400 four-star ones, compared to 1,000-plus three-star recruits and at least as many lesser ones. In other words, the top recruits face a ton of competition.

    While we agree this observation is statistically accurate, is it practically useful?

    Why does the draft success rate fall from about 50% for five-star players to less than 10% of four-star players? What does this extraordinary decline say about the quality of the four-star assessments?  Why is there such a huge gap in NFL draft outcomes when the services’ raw score differential between five-star and four-star players is often no more than a tenth of a percent?

    Similarly, because the draft odds for two-star and three-star players are so incredibly low, is having a three-star rating more helpful in assessing an NFL player’s draft probability than a two-star rating?

    Along those same lines, is a player a “can’t miss” five-star prospect because a recruiting service says he is, or could virtually anyone with a pulse watching a high school football game come to a similar conclusion?  

    Even recruiting service defenders wonder if the recruiting services assign grades based on the services’ proprietary analysis, or if the services merely base grades on the quality of teams pursuing the recruits (i.e. players recruited by large schools such as Alabama are normally graded higher than players pursued by smaller schools).

    If anyone can define a five-star prospect (and we believe anyone can), and if there is little discernable difference between the ratings of three-star and two-star players (and we do not believe there is a significant difference), why assign recruiting grades at all?  

    It is not clear to us how college programs may have benefited from relying on the assessments of recruiting services given these services’ extraordinarily high failure rates.

    Colleges were recruiting great players long before recruiting services came on the scene.  The best of these players were on everyone’s radar screen, with many such players gravitating to the same schools they do today. On occassion those players lived up to their billing.  On occassion those players did not.  Meanwhile, coaches pursued less talented recruits to help fill out rosters,  just like they do today. These coaches likewise missed out on a large number of diamonds-in-the-rough, just as they do today. 

    This is not to say that recruiting services do not have some function. At best, they offer a useful information source that allows coaches to compare measurable attributes such as speed, strength and size, but that appears to be the extent of their utility.

    These observations render the idea of issuing recruiting rankings by team and by player a perhaps enjoyable, but ultimately meaningless exercise.

    In Defense of Recruiting Services

    Recruiting service defenders point to recent BCS champions, all of which have been successful attracting what the recruiting services describe as blue chip talent.

    Does this observation suffer from confirmation bias?

    What the defenders leave out is that many more schools routinely score highly in these league tables, yet underperform on the field in spite of their talent.  Most recently these programs include schools with supposed top five recruiting classes like Texas, Florida, Michigan, USC, Tennessee, Florida State, and Notre Dame.  

    These teams attract what the recruiting services believe to be the best players in the country, and then subsequently fail to perform against what is presumed to be inferior talent.  The evidence of teams underperforming in spite of their talent is at least, if not more, pervasive as is the evidence of schools recruiting well and performing at a high level. 

    Recruiting service sympathizers note that projecting future NFL draftees based on these players’ high school performance will naturally be imperfect, but they forget an important point.  

    Recruiting services operate in rarified air.  They evaluate the uppermost echelons of college football talent.   The standard by which we are judging the services is not based on an “average” recruit.  We would, of course, expect the probability of an average player being drafted to be remote. We are judging the services on their assessment of the top 30 players per year out of a pool of over 300,000 eligible high school players in the country (the top 0.01% of all players each year).

    We are evaluating the services on what should be equivalent to a wide open layup, but observe first round results that are slightly better than the odds of hitting a half court shot. 

    If the recruiting services can’t deliver results at this level, what does that say about the recruiting services’ evaluation process overall?

    Which side is right?

    NFL teams select around 254 players in its draft each year. These draftees represent roughly the top one percent of all eligible NCAA Division I college juniors and seniors.

    Among the 253 players selected in the 2012 NFL draft, just 11 draftees, including four first round picks, were considered 5-star college prospects (4.3% of all players taken). The majority of players (62.5%) were considered 3-star and 4-star prospects, while over one-third of selectees were considered 2-star prospects or were not graded at all.

    2012 NFL Draft by Recruiting Rating

    These outcomes caused us to wonder, “what happened to the other five-star can’t-miss prospects?”

    For that, we go back to the classes of 2006-2008 to find our answer.

    Career Outcomes for Non-Drafted Five-Star Recruits (2006, 2007)Over the three-year period from which these current NFL draftees were recruited, 81 players were deemed five-star prospects. To date, 47 of these players have been drafted in the NFL, including 13 number one draft picks.

    When we look at the class of 2006 and 2007, the NFL drafted nearly two-thirds of the players Rivals.com defined as five-star college prospects.

    Many of the undrafted five-star recruits had successful college careers or continue to keep their hopes alive as free agents or by playing in some other professional capacity.

    About one-quarter of these prospects failed to live up to expectations. Some never displayed their promise at the college level, while others were waylaid by academic and disciplinary problems.

    There is No Such Thing as a Football Factory

    In evaluating the recruiting outcomes for five-star prospects, one key observation jumped out at us: no single college has a unique ability to produce NFL talent.

    2012 NFL Draft by College

    In the 2012 draft, 253 players were selected from 103 colleges. Alabama had the most players taken, but these eight players represented just over 3% of the entire draft class.

    Similarly, when we look at the draft outcomes of five-star prospects by school (173 five-star prospects attended 41 schools from 2002-2007), we can see that while USC stockpiled the most 5-star talent, only half were drafted, and just 15% were first round selections.

    Five-Star Recruit Outcomes by College (2002-2007)

    Florida, led by now-departed coach Urban Myer, was the only school to demonstrate a meaningful ability to help develop five-star prospects into NFL draft picks.

    These statistics lead us to the conclusion that the idea of a football factory is the stuff of fantasy (in which the imaginary factory produces no meaningful outputs). 

    Similarly, since so few players actually make it to the NFL, and no school can claim a unique ability to produce NFL draftees of any significant quantity, assessing the quality of a football program by the number of NFL players it generates is also, well, pointless.

    Final Thoughts

    Getting into the NFL is the ultimate meritocracy. It does not matter which school a player attends. If a player has enough talent and determination, NFL scouts will identify him. Not all players may be drafted. After all, there are only 253 or so draft spots. But the best players, barring injury, will be able to forge careers in the NFL regardless of which schools they choose to attend.

    This understanding should help change the manner in which college athletes assess where they ought to go to school. The results reveal the following:

    • No single program or coach has a unique ability to get a player into the NFL
    • Going to a winning college program does not improve the odds of entry into the NFL
    • Since most players require one or two years of physical development before they are able to contribute on the field, they are better off identifying programs where they are most likely to start or contribute significantly by their sophomore season.

    Joining a talent-rich team is almost the surest way to never see playing time.

    Since the odds of making it to the NFL are so remote in the first place, our advice is that prospective recruits ignore the rankings and attend the school where they can develop into successful college players and make the most of the academic opportunities afforded to them.  

    Just ask Chandler Harnish.

    ******************

    Methodology

    There are at present four recruiting services of note: Rivals.comScout.com24/7sports.com and ESPN Recruiting Nation.

    In examining the different services, we find that all represent their findings with star ratings, grades and commentary that are broadly similar.

    We chose to focus on Rivals.com for a reason that we believe is complimentary. Rivals.com has developed robust database and search tools that the other platforms, in our view, lack, making our ability to conduct research far more efficient.

    We examine recruiting outcomes for seven recruiting classes from 2002-2008. In some datasets, we excluded a detailed analysis of 2008, because many of those players will be completing their eligibility in the upcoming 2012 season. We exclude the classes of 2009-2012 because almost all of those players remain in school.

    We only focus on high school players in our assessment, and do not include five-star rated junior college or prep school prospects.

    PrintView Printer Friendly Version

    EmailEmail Article to Friend

    References (1)

    References allow you to track sources for this article, as well as articles that were written in response to this article.
    • Response
      The program matters, and it matters a great deal. The choice of the right (or wrong) program can double a recruit's chances at the NFL, or it can halve them! And in the special case of Duke, bury them completely. The programs at the top - Iowa, Cal, USC and Ohio ...

    Reader Comments (1)

    This has nothing to do with sports recruiting services working or not.

    February 13, 2013 | Unregistered Commenterbasketball resume.com

    PostPost a New Comment

    Enter your information below to add a new comment.

    My response is on my own website »
    Author Email (optional):
    Author URL (optional):
    Post:
     
    Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>