Friday, August 11, 2006

WPA and Overcrediting Relievers

I don't find WPA as interesting as most of the saber-oriented internet community seems to at present. I don't think it's worthless, though. In any event, I've noticed that a very common criticism is that the distribution of credit is heavily skewed toward relievers. While I understand the sentiment and I've read several discussions regarding this point, the argument that I don't recall having seen is that this criticism simply isn't unique to WPA. Run average and ERA already do the same thing, no? It is much easier to pitch in relief (or, rather, it is much easier to be used as major league relievers are used than to be used as major league starters are used; a starter who pitches for one inning would have an easier go than a reliever pitching innings two through eight). Any metric which does not compare relievers to relievers and starters to
starters (and hitters to hitters) will 'favor' relievers.

As far as I can tell, all of the WPA compares relievers, starters, and hitters to the same baseline of R/PA, and since the task for relievers is the easiest of those three, they will come out on top, just as they will in nearly all statistical categories. Or at least, all of the models that use theoretically-derived WE tables seem to apply the same run environment to all players; BP or maybe somebody else might be using empirical WE data, which has its own set of sizable flaws, and BP probably does all of its WE stats against a replacement level, which negates all of the benefits of using the empirical values anyway.

Now, WPA does 'favor' *some* relievers relative to all other players in that a reliever finishing a game needs only succeed in walking away with the game to receive a positive score: a closer who enters with a three run lead in the ninth, walks the first five hitters, and finally induces a triple play groundout will receive a positive WPA value despite pitching very poorly. But so what? Similar quirks exist in any baseball statistic I can think of, and it's not WPA's fault that people treat it as if it were a pure measure of value. There are infinite ways to measure value in baseball, and expecting WPA to be a reasonable stand in for all of them is silly. WPA is Win Probability Added, not Player Goodness. So while WPA's usefulness derives from adding more context to the raw numbers, many of the criticisms of it derive from people's willingness to strip WPA of its context. As with any other metric, it will not tell you anything meaningful about player value across contexts; WPA cannot possibly prove that Jonathan Papelbon has been more valuable than Tim Wakefield. It can only tell you that, comparing the two to the same run environment, Papelbon has added more win probability. Anytime numbers are removed from their context the ensuing comparison will be 'unfair.' So if the problem you have with WPA is that relievers who pitch poorly with a lead in the 9th get more credit than hitters who jack doubles in the fourth - well, why are you even using WPA to compare them in the first place? The only comparison WPA can make is Win Probability Added.

RBI is not a flawed statistic because it credits Joe Carter more than Bobby Abreu; people who use RBI to argue that Carter had more value are flawed. The same holds for WPA.


Post a Comment

<< Home