I made a top 3625 prospects list recently.
It wasn't that hard. I had previously made crummy minor league stats leaderboards for each affiliated league by using PA and IP cutoffs and then calculating some simple Z-Score sums for each player, based on the input statistics for the no-longer-available KATOH system.
Basically, players get positive scores if they were better than league average at the things that correlate with MLB outcomes according to Chris Mitchell's articles about KATOH. For hitters, these stats are Age, K%, BB%, ISO, and BABIP. For pitchers we care about Age, K%, BB%, HR%, and Games Started % (there were a couple of other inputs in KATOH such as pitcher handedness or SB attempt rate for hitters that I chose not to use for simplicity).
To turn my league lists into the top 3625 prospects behemoth list, I just applied a mild penalty to every pitcher, for being a damned pitcher, and then mild penalties for prospects in leagues that are further away from the Majors. These penalties were small enough to really just be tie-breakers. If a hitter ended up close to tied with a pitcher, the penalty would give the edge to the bat; if a player ended up close to tied with a player from a lower level, the player with more proximity would rank higher.
The big list itself is not that interesting. It identifies some under-appreciated sleeper prospects (draft Nate Lowe in your dynasty leagues!) but the exercise is crude. The list is bad because prospects that didn't make the PA or IP cutoffs don't even appear, there are no adjustments for position or league strength, and dozens of other reasons. What the list does do is put the performance of prospects in context... but what context?
I think the shape of the distribution of the scores is interesting. I think it is an illustration that can tell us a bit about how much we should care about baseball prospects of varying talent levels. I also think the figure has general applicability to traditional scouting methods and rankings.
Here is what the plotted scores of the top 3625 prospects looks like:
Of course that's what the shape would be, because the cumulative scores are based on Z-Scores, which are based on standard deviations. For any given skill, if normally distributed, 68% of players will be within one standard deviation of the mean. Only 5% of players will fall outside of two standard deviations for any skill. It makes sense that when we add up Z-Scores for projectable skills the majority of minor league players will fall pretty close to zero, and a relatively small number of players will have very positive or negative scores.
All we really care about is the positive tail of this distribution. This is where we'll find most of the future MLB players.
The shape of this curve is what is important - it gets steeper at higher ranks, to the left. Steeper means the differences between adjacent prospects starts to increase. The difference between Vlad, the #1 prospect, and Eloy Jimenez, the #4 prospect, is about the same as the difference between Eloy and the prospects ranked at the 150 mark.
During prospect list season a lot of fans enjoy chirping the prospectors for leaving certain players off their top 100 list, or their organizational top 10 list. This is clearly silly.
There is nothing special about the number 100, it was just a natural choice for list production because it's a round number and going that deep tends to be onerous enough for the people who do it. By projectable talent, though, there is little difference between the 100th best prospect (Dan Vogelbach's AAA numbers, by the way) and, say, the 250th (Matt Wisler). Don't ever get mad about top 100 snubs.
You should view team lists with the shape of this curve in mind, too. There is not really a huge projectable difference between a typical team's 10th best prospect (#300 or so on this curve) and their 30th best prospect. I would go so far as to say that it's almost silly to try to rank organizational prospects after the top 10 or so - it's an expression of specificity that simply does not exist. Yes, I know we just did a Blue Jays top 50 list last month. It became apparent to me during that exercise that trying to order prospects down the list was rather foolish.
Things that you might want to pay attention to in future prospect list perusal would be:
the players that the prospector is putting at the very top end of their list - they are signalling that these players may be on a different type of talent curve;
the order in which the prospector's top ranked prospects appear - statistically the differences between elite prospects can be large, so an opinion that Tatis Jr. is better than Guerrero Jr. is not trivial;
the players that the prospector is ranking controversially low - they are signalling that said players are not special. This may be the case even if that dude still appears on their top 100 list. If you look at the second image above, you'll see, generally, that prospects in the top 50 or so are on a much steeper talent curve than prospects in the 50-100 range.
Moving away from prospect lists, the final point here will be what the curve illustrates about actual player development. Remember that these scores are based on the player's Z-Scores in real skills - things like ISO and K rate. The scores are additive. The small differences between most prospects means that developmental steps by a prospect in any projectable skill will lead to massive increases in their theoretical prospect ranking. Imagine that a pitcher with a score of 0.000 were to spend a summer at Driveline, improve his velocity and design a new pitch, and turn his average K rate into a K rate this is two standard deviations above average. His prospect ranking would improve from about 1650th to something like 450th through a jump in that one projectable skill, being K rate, all else being equal.
I think the moral of that example is essentially: if you notice during the season that an unknown prospect is putting up significantly better numbers in one or more of their projectable skill indicators, don't be overly hesitant to shoot them up your mental rankings, especially if you can find some sensible narrative supporting their improvement. It doesn't take a quantum leap for actual baseball prospects to move from outside their organization top 50 list to just outside their organizational top 10.
Cover image source is Joel Dinda on Flickr:
Thanks for reading "Perspective on Top Prospects Lists" by Nick Hill. If you have any questions or comments relating to this article, we encourage you to leave them below.
For all general inquiries, we can be reached at the following:
Twitter: https://twitter.com/radio_scouts
Email: radioscoutspodcast@gmail.com
Comments