NFBC Message Boards

Every Monday until I run of things to say or get a better offer, I’ll be posting an essay that in some manner, way, shape or form might have a speck of interest to a fantasy baseball enthusiast.



Recent archives:



1-29 Ladies and Gentlemen, Start Your Engines



1-22 Ch, Ch, Changes



1-15 Whither an Expert



_______________________________________________________



Every year at this time, I am asked the same question, “Can you show me why your site’s projections are better than anyone else’s?” And every year at this time I answer the same way, “No.”



In short, I have yet to either find or invent a means to compare projections accurately. I know several people have tried, some as a means to pimp their own and some as an unbiased third party. I have seen countless attempts but they all fall woefully short and are more misleading if not disingenuous than anything. The mathematics deployed is often quite elegant and requires an advanced degree in statistics, but the end result is always the same – pure hooey.



There is so much more to a projection than just the number and no comparative analysis can account for everything involved. For example, let’s consider something as seemingly innocuous as the home run. There are several elements to consider when looking back and determining how well one did at predicting the number of home runs a player hit. The stat itself is actually a combination of two factors – the rate of performance (HR per at bat or plate appearance) and the amount of performance (number of at bats or plate appearances). Is simply nailing the raw stat indicative of a successful projection? What if you actually predicted the player to get fewer at bats so the only reason you were correct is he enjoyed more playing time than you anticipated? That is, you incorrectly projected the rate of performance. I’m not sure I consider that a success. Then there is the whole luck factor. A little gust can turn a couple of would-be doubles into homers. Or the opposite might be true where a stiff breeze knocks down a couple of majestic would-be jacks and they are caught at the wall. There is enough chance involved that some projections might be affected.



I just have not encountered a method to measure all that is involved, so instead, I concentrate on the method the prognosticator used as opposed to the strict result. I have this saying – I’d rather be wrong for the right reasons than right for the wrong reasons. The reason is thus. Over the long haul, if the methods I use are sound, I will be right more than I am wrong, and that is what counts. Let’s go back to the home run example. I project a player to hit 25 homers in 550 at bats. Someone else pegs him for 18 in a similar number of at bats. The player’s season is cut short via a freak injury and he ends with 18 knocks in 400 at bats. I had the rate of performance just about perfect, but could not have anticipated the season ending injury so my projection was wrong, but the other guy was spot on, only because the player missed about 150 at bats. Maybe the answer is in looking at the rate of performance, but I have not seen a method I trust. Anyway, the point is who is more likely to get next year’s projection correct? My money’s on me as at least I got the rate correct.



This doesn’t even begin to consider all the noise involved with examining rate stats like batting average, ERA and WHIP. Anything that involves the number of hits has a difficult time discerning the luck aspect from the skill aspect. This is a topic I plan on covering later in this series of essays, so I will save the crux of the discussion for that time. But suffice it to say a player can exhibit an identical skill set two consecutive years and have his batting average differ by as many as 30 points.



Luck is just that, out of our control and not predictable. No projection system can account for fate. I’d rather focus on the skills aspect. To me, the best projection procedure concentrates on skills, converting those skills to stats. It is not all number crunching, as scouting is part and parcel to analyzing trends. Will a three-year positive trend continue to improve or will it regress? I’m not talking about going out on a limb with a gut call, but justifying a skill by watching a player.



So while some blindly believe a particular prognosticator is better than others based on fallacious evidence, I’ll spend my time understanding the methods employed. Because I’d rather be wrong for the right reasons than right for the wrong reasons.



[ February 06, 2007, 12:03 AM: Message edited by: ToddZ ]

It is amazing how we fantasy players takes these projections [which everyone realizes are merely an educated guesses] and then apply various forms of precise mathematical formula to them in order to obtain a numerical value. This can be a rating score or an auction dollar amount. I admit I am as guilty as the next man.



All these projections can really provide is drafting guidelines. They help you discover players on the uptick they may have been overlooked. They can keep you from drafting a player based on name and past accomplishments rather than his true current skills. Hello, Sammy Sosa and your buddy Juan Gonzalez.

Good point Mike - I also get a kick out of how some will display this dollar value to two digits as if it is correct to that place.

In my opinion the most important stat is number of at bats, the rest is relatively easy.

To an extent I agree with ABs. I look at it as games played, i.e. health. Every time your starter is out for an extended time, you have to replace him with someone from your bench and there is never a bench player who has better numbers - else he would be active.



I would be more than willing to drop five spots on my list in each round if you could guarantee me that the player I picked would play in 158 or more games. I might give Pujols a bit more leeway but I think that is about it.

I admit I'm resistant to some of the things in this thread. For example, I think finding a correlation coefficient for a sample of players with x number of bats (maybe 400) is a good way to measure the accuracy of that part of a projection system. Now, it doesn't do much for, say, stolen bases, but it covers several stats and gusts of wind don't make much difference if your sample is big enough.



The term 'educated guess' is correct for some systems and some people have luck with that type, but it is far from the only one.



Finally, I don't understand the resistance to precise stats. Whether this is in terms of decimal places or batting average, I don't get it. For example, let's say you expect a degree of error in you ba projections of +/- 0.020 and you have a .294 average projected for a guy. Why are you going to say, "He'll hit about .290"? Now instead of =/- 0.020, you have + .024 or - .016. It makes a lot more sense to be as accurate as possible. Over the course of 14 hitters, you have a better chance of them evening out if you aren't doctoring them all the time.



[ February 06, 2007, 01:22 PM: Message edited by: bjoak ]

When I say decimal places, I am referring to dollar values like so and so is worth $18.92.



Frankly, if they are presented that way, it reflects the person does not really understand valuation theory.

Well, I don't use auction values in a snake draft because those numbers don't mean anything in a snake draft (my values are according to points in the overall competition), but if I did, I'm not saying that decimals are all important, but if I was going for a shortstop and I felt exactly the same about two of them, why wouldn't I want to know which one came out better, even if that amount is small enough as to seem negligible, it makes some sense as a tie-breaker. And I know if your values were slightly different the other one could come out better, but again, like with my BA argument, you're still more likely to have the higher one be worth more than the lesser one.



Frankly, if they are presented that way, it reflects the person does not really understand valuation theory.

Why does how they are presented reflect the person's understanding? Maybe they were too lazy to change them, or they don't know how to put a whole number on a spreadsheet or just want to see exactly how the guy came out. Again, the difference of 49 cents, I have no idea, but is probably 5 RBI's or something. No one projected those out of thin air. Maybe he'll hit them or maybe he won't but it seems silly to take the guy who is projected lower or to not at least know which one came out better. But to laugh at someone just because they decided to show a different number of decimal places shows that you take that value too seriously.



[ February 06, 2007, 02:42 PM: Message edited by: bjoak ]

Todd, I find it funny that you take these shots at Baseball HQ and Ron Shandler about player values using 27.02 vs 27. But than say you can not say your web site is better on projections.

Ron Shandler explained this last year about projections. You seem to have read this and used it as your own. Though Shandler's projections come out early it seems to be a driving force for the rest of the web sites to try and chip away at them. Capt Hook took a shot about thier stright draft advice. But Capt Hook hasn't won it either.

A value based on projections that are +/- a whole number can't be accurate to 2 decimal places. To me, this reflects on the credibility of the presenter. Why would I rely on someone too lazy to make the correct mathematical adjustment or that does not understand how to consider error when doing calculations?



But to laugh at someone just because they decided to show a different number of decimal places shows that you take that value too seriously.

I certainly have been accused as such. But I'd rather resemble that remark than provide information without taking the theory and process serious enough.

Todd, I find it funny that you take these shots at Baseball HQ and Ron Shandler about player values using 27.02 vs 27. But than say you can not say your web site is better on projections.

The shot wasn't taken at anyone in particular.



Ron Shandler explained this last year about projections. You seem to have read this and used it as your own. That's a pretty strong accusation John, and quite frankly it is false and out of line. All you have to do is search my message board and you will find a plethora of threads on how we as a group feel about projections. These threads far pre-date Ron's writing on the subject. It was his theme for the First Pitch Series a few years back, in fact.

Good thing you are not ranking players on the content panel:



"Greg, I have first basemen # 4, 5, 6, and 7 marked down for the same $ value so please score them all as 5.5 on my sheet. If I actually ranked them it would prove that I don't know anything about valuation theory."

Bjoak, I'm asking this because I am honestly curious, not to be a wiseass. Personally, I learn more from discussions like this than anywhere else.



How would you rank the following 4 players with these respective dollar values?



Player A: $20.32 +/- $6

Player B: $19.78 +/- $4

Player C: $20.49 +/- $3

Player D: $19.69 +/- $3

Todd, What other site puts thier values beyond two digits? Of course you were taking a shot at Shandler. As far as who was first about saying there is an error margin in projections?

John, there are several sites that do it. If you REALLY want to know, e-mail me and I will send you a list.



As for a margin of error in projections, it is common knowledge, bordering on obvious. We talk about it in the old Mastersball Annual, a book we published for a few years before we went exclusively to the web. This was six, seven years ago.



Again, do a message board search, and you will find it is an ongoing project of ours to attempt to put an error bar on projections and the ensuing value. Some stat projections involve greater variance than others -- we are tying this into game theory and also to help explain the typical hitting to pitching split commonly used in auctions.



John, I don't know what your agenda is here, but I find it quite ironic. Truth be told, the reason this essay is not up to snuff, as bjoak has pointed out, is I had another piece ready to go, but I decided last minute not to run with it. The piece defended Ron in his decision not to participate in the NFBC, something that has been discussed in other threads. Ron and I are friends and I frankly don't like what I feel is a misconception with respect to his not playing. But before posting, I thought about it and concluded Ron is a big boy, he doesn't need my fat ass defending him. Dollar to a donut he is well aware of the accusations on these boards and has opted to not respond. I figured I owed him the courtesy to do the same. Until now, of course. Sorry Ron.

Ron explained here last year why he doesn't play in these leagues. If this issue still gnaws at anyone, just use the 'search' feature and look it up.



Next. Move on.





And there are many sites that use two decimals.

I guess I understand you better now. You are focused on risk management so the precise values don't matter at all in the face of that. I still think it's inadequate to judge somebody just because they choose to show a more specific number.

I have a bunch of values for players in my computer right now carried out to the ninth decimal place, not because I regard what the value of a player is to the billionth place but because I didn't have a reason to shorten them. It really doesn't make a difference.



But that's what I am asuming you're talking about with +/-, the risk factor. My answer is that I don't worry about it. Health I worry about, but I don't worry about volatility in projections. I figure if your projections are good enough you will come out ahead in some and behind in others. It all works out. Or at least it works out better than it would playing it safe all the time. It works for me.



Also, my projections are not all that volatile to begin with.



[ February 06, 2007, 04:29 PM: Message edited by: bjoak ]

Todd, My agenda is to provoke a response from you that is not planned in advance. If you come on a message board promoting a web site like fantasybaseball than your also open to probing questions. My questions are based on what I know right or wrong. But sometimes your site is wrong

also. Ask Capt Hook about the piece he wrote last year about the $650 NL auction league. He went into detail how the young lady that was a mom was in over her head. The young mom finished ahead of Capt Hook's team. I didn't see a followup on that one. By the way she's back! Look out Perry!

Todd, My agenda is to provoke a response from you that is not planned in advance. If you come on a message board promoting a web site like fantasybaseball than your also open to probing questions. My questions are based on what I know right or wrong. John -- I have NEVER shied away from probing questions. But you did more than ask a probing question, you accused me of taking Ron's work and using it as my own. That is not a probing question. If that is your means of provoking an unplanned response, you really need to work on your bedside manner.



If you want to call us out for being wrong, then just do it. You've done it before and so long as you have valid reasons, that is your right as a consumer. We don't deny that, we never have.



Now, with respect to my remark about sites using two decimal places, I apologize. It was a gratuitous shot that served no purpose and did not belong on these boards.

Originally posted by Walla Walla:

Todd, My agenda is to provoke a response from you that is not planned in advance. If you come on a message board promoting a web site like fantasybaseball than your also open to probing questions. My questions are based on what I know right or wrong. But sometimes your site is wrong

also. Ask Capt Hook about the piece he wrote last year about the $650 NL auction league. He went into detail how the young lady that was a mom was in over her head. The young mom finished ahead of Capt Hook's team. I didn't see a followup on that one. By the way she's back! Look out Perry! John, not sure what you were reading but here are the only two references to Katherine Eckert in that article:



1)"For one thing, the beautiful young mother that was drafting on our right had only purchased two players and had more money than a few teams put together. This would likely get nasty as I doubted she would be able to spend her money effectively and thus be in my way among others later in the auction."



2)"But back to the indecision in the end game. I emerged from my hibernation to find Mrs. Moneybags filling in her roster, but not being quite aggressive enough given the situation – WHEN you have money control, it won’t do you any good if you don’t TAKE advantage of it!

You should be able to see the players you want and GET them. When you back off a fight on a player or two, you will just get pushed that much more as everyone has to fight for a player or two with the big stack."



Show me where it says she was in over her head!



Of course she finished ahead of you as well.



Go ahead and join us in that league again and see if you can beat either one of us.

Todd, you are right on with respect to specificity on valuations. I have detailed models on player values, and far more detailed models for valuing businesses and markets.



I can't help but think that certain analysts just don't get it if they discuss the output of valuation equations or rankings as if they are sacred or magical numbers. Player values are akin to "price targets" in the stock market, and many analysts/investors will say for example, "if this stock reaches $50, I will sell it, when in reality they would be better served by a plan to sell or buy within an appx range, with other factors such as prevailing risk and opportunity being the deciding factor for whether they should sell at $47.50 or hold out for $52.50.



This is the same reason, I believe, that having a cheat sheet with dollar values or rankings should be used as a guideline during a draft. If it comes down to two players with a marginal difference between them, choosing the one where confidence level or risk/opportunity is better is likely a wiser choice than choosing the one that is mathetically modelled with $0.60 advantage.

Wow, a lot of hurt feelings around here.



I'd like impart my thoughts on the topic of projections. I too have a value method similar to Bjoak's that provides values based on the number of points avilable under the NFBC format (ie 75 points for both Batting and Pitching). For example, Pujols value equates to approx. 19 points of value, meaning he alone will get you 19 of the 75 batting points. Keep in mind this system breaks down each and every category into values...so your goal is 15 points in any one category.



Now of course, this all depends on the accuracy of your projections. I have used the same 7 publications over the past 2 years: none of which are Pecota or Shandler. Call me cheap! Anyway, I have tested their accuracy vs each other and vs an equally weighted average of all 7.



And I will say this, some are better than others. With batting there are a couple that haven't done so well each of the last two years. The other 5 have had both good years and average years. The pitching projections is where I see the biggest difference. A lot of them (4 or 5 of them) haven't done a very good job of it over the past two years. There are 2 or 3 that have done a good job.



So from my view there are several advanatges to value projections:



1) finding undervalued players;

2) helps you appropriately draft/stack each of the categories (while I draft) - here is no point in drafting more HR's than is need for 15 points. I can also track other managers in my league during the draft, which gives me an indication on where they may be heading next. Even if that helps me in getting just one or two players, then its worth it.

3) rate publications for next year's draft (ie assign different weights)

4) more accurately predict draft order of players as it takes into consideration a number of projections, thus representing the league better than any one publication

5) value mid-season pickups; and mean reversion analysis

6) allows me to adjust AB's and IP's and revalue at the drop of a hat (ie 10am on draft day)





Now if only the players would always match their projections. Now this kind of stuff isn't for everyone as many successful managers have won money without such precise analysis.

I find this thread really kind interesting. Todd put his thoughts out there about being wrong for the right reasons. I think he will find a year where the right reasons will give him the results if he can tie enough of those right players together.



I see John is defending Shandler. I know he has to be a fan of him, but I also know he belongs to fantasybaseball.com. My best guess would be that he get usefull information from both areas.



I also belong to fantasybaseball.com. I signed up last year when Perry came on board. I have a lot of respect for him as a player. We have played against each other in the AL auction in Vegas. He has finished second twice in three year. I beat him out the first year by a hit or an out. I'm not sure which, but I know I was lucky to win. I consider him a threat each year. As for his other results, I'm sure when it is all said and done he will have his share of victories.



Both Todd and Perry have a lot of passion for baseball. I have enjoyed their work. I will continue to listen to their views. I think Todd has done a great job on his post here.

KJ, I agree with your points. I was just reacting to Todd's admittedly harsh comments about anyone who uses decimal places. There are any number of other factors that I consider, especially when the numbers start getting close and as I said the difference between close guys is usually negligible. But I'll admit that I don't have voluminous nformation on some guys. Last year, I was desperate for a second baseman and I had to choose I think Jose Lopez or Junior Spivey neither of whom I had ever considered and didn't know much about other than what was in front of me. Lopez had the better number and it was, okay, this guy. That was probably the only situation where I did that, but I don't see why I should make an effort to hide that information from myself. Looking at the extra numbers doesn't bother me. The fact that someone is walking behind me and assuming that I am bad because of nothing other than how many decimal places I am using is disturbing. I don't even see a remote connection between the two.



Everything I just said notwithstanding, if it doesn't hurt you in another way, getting value is never bad. If you save 60 cents on every player in an auction, that might be negligible, but the 12 bucks you saved (or, for that matter, 12 bucks worth of stats) after 20 rounds is not.



[ February 06, 2007, 09:24 PM: Message edited by: bjoak ]

Great thread. Bjoak, your last post should have ended with the completion of your first paragraph. Great points. But when you in essence say that decimal point utilization buys you 12 dollars worth of stats, it more or less contradicts what you just finished saying.