So it looks like once you get to approximately the 3600 shot mark, you can start to be reasonably certain of goalie production (between ~.905 and ~.922 on the chart). However, we run into the same problem that we run into when trying to predict player skill - what if their skill level changes over time? The longer it takes to assemble the appropriate sample size, the less reliable that data is. 3600 shots works out to about 2 full seasons. Are goalies really the same player after 2 years that they were at the start? Perhaps in some cases.
I think the big takeaway from this is something I've been saying for a long time. Over the course of 100 shots (ie. one week of fantasy play) ANYTHING can happen. The goalie can post .80 or 1.000, and any attempts to predict week-to-week performance is a coin flip. So in head-to-head leagues, should you attribute much value to goalies? I don't think so. At the draft it is better to spend your picks targetting more consistent categories like FW and hits, and then cross your fingers that your bad goalies have a flukey season. (I won 3 fantasy years in a row after implementing this approach, and when I changed my plan last year and picked up a bunch of stud goalies, they invariably were a bust and I lost)
Thanks for the article!
@Gnial - while anything can happen over the course of a week in fantasy hockey, the coin Henrik Lundqvist is flipping has a different weight than the coin Mike Smith is flipping.
This sort of analysis clearly demonstrates how many shots are needed before a career sv% stabilizes and can be taken as an accurate measurement of the goalie's skill (by that measure), and also the expected variability given a specific sample size. Pretty cool to see - and why I come to LWL :). It'd also be pretty useful to see if certain goalies are under-valued because they had a "bad" year that falls within the expected distribution for the number of shots faced for a single season. You might also get an idea if coaching or team changes have really impacted a player, or if it's just the usual variability of the sample.
I'd be interested to see how the distribution/standard deviation around the mean change with variable career sv%. What's your plot look like for a .900 goalie, and a .920 goalie? Basically, can you determine how big a sample is needed before being able to predict the expected career sv% mean (correlating the distribution of a small sample size to the cumulative mean)? It may not be useful (perhaps too many shots are required), but it'd be worth looking into. Even a small correlation would be something.
It'd likely be very helpful to track veterans season-to-season to see when they start to have higher variation then you might expect (a sign of decline) or when the distribution starts shifting towards being predictive of a lower career sv%. Maybe just a control chart sort of plot of a moving average (x shots, maybe 1800, maybe more) would be useful in that way (with a line for the career sv%, and a band for the distribution around that mean given the averages sample size).
Leave a comment
In order to leave a comment, you must be a registered user at our website and logged in.
So it looks like once you get to approximately the 3600 shot mark, you can start to be reasonably certain of goalie production (between ~.905 and ~.922 on the chart). However, we run into the same problem that we run into when trying to predict player skill - what if their skill level changes over time? The longer it takes to assemble the appropriate sample size, the less reliable that data is. 3600 shots works out to about 2 full seasons. Are goalies really the same player after 2 years that they were at the start? Perhaps in some cases. I think the big takeaway from this is something I've been saying for a long time. Over the course of 100 shots (ie. one week of fantasy play) ANYTHING can happen. The goalie can post .80 or 1.000, and any attempts to predict week-to-week performance is a coin flip. So in head-to-head leagues, should you attribute much value to goalies? I don't think so. At the draft it is better to spend your picks targetting more consistent categories like FW and hits, and then cross your fingers that your bad goalies have a flukey season. (I won 3 fantasy years in a row after implementing this approach, and when I changed my plan last year and picked up a bunch of stud goalies, they invariably were a bust and I lost) Thanks for the article!
@Gnial - while anything can happen over the course of a week in fantasy hockey, the coin Henrik Lundqvist is flipping has a different weight than the coin Mike Smith is flipping.
This sort of analysis clearly demonstrates how many shots are needed before a career sv% stabilizes and can be taken as an accurate measurement of the goalie's skill (by that measure), and also the expected variability given a specific sample size. Pretty cool to see - and why I come to LWL :). It'd also be pretty useful to see if certain goalies are under-valued because they had a "bad" year that falls within the expected distribution for the number of shots faced for a single season. You might also get an idea if coaching or team changes have really impacted a player, or if it's just the usual variability of the sample. I'd be interested to see how the distribution/standard deviation around the mean change with variable career sv%. What's your plot look like for a .900 goalie, and a .920 goalie? Basically, can you determine how big a sample is needed before being able to predict the expected career sv% mean (correlating the distribution of a small sample size to the cumulative mean)? It may not be useful (perhaps too many shots are required), but it'd be worth looking into. Even a small correlation would be something. It'd likely be very helpful to track veterans season-to-season to see when they start to have higher variation then you might expect (a sign of decline) or when the distribution starts shifting towards being predictive of a lower career sv%. Maybe just a control chart sort of plot of a moving average (x shots, maybe 1800, maybe more) would be useful in that way (with a line for the career sv%, and a band for the distribution around that mean given the averages sample size).