WAR and Projections vs. Salary

Sky Andrecheck recently looked at how much players are paid based on their actual wins above replacement. He came up with a few estimates between 5.4 million per WAR to over 7, depending on what equation he used. In any case, this is much higher than 4 to 4.5 million commonly accepted in the blogosphere. Sky was kind enough to send me his list of players, along with their Fangraphs WAR. I used Cot's contracts to find average annual salary for these players, and my own site to see how many WAR I had projected for 2009. The players used in this study are those who signed a contract after having 6 years of service time, we are only looking at the free agent market.

Here are some of the results:

  1. The sum of the player average annual salaries was just over 1.5 billion. They contributed 253 WAR. Simple division means they got 5.95 million per WAR.
  2. I divided the players by year signed. I grouped all players signed before 2007 together since there weren't many contracts still in force in 2009 from that long ago. These players got 6.97 per WAR. 2007 was an awful year for free agents. Gary Matthews, Juan Pierre, Carlos Lee, Soriano, and Zito are a few who make teams wonder what they were thinking. This year group got 8.11 million per WAR. The 2008 class got a more reasonable 6.2 million, and the players who signed for 2009 got 4.82. This number is still surprisingly high.
  3. I compared WAR to projected WAR and found that, across the board, my projections were too optimistic. This is surprising to me, but the data do not lie. Check out the projections on Fangraphs. The Bill James projections have been notoriously optimistic for years. My projections (and also Dan Szymborski's ZIPS) are usually considered the realistic, or even pessimistic ones. Turns out we're only realistic in a relative sense. For this group of players, I had projected 301 WAR, they only delivered 84% of that, or 253.
  4. It is true that my projections show too much playing time. For example, I have most minor leaguers and part timers forecast around 300-450 plate appearances, even though they likely will not get that much time. So if I'm forecasting a bench player for 1.5 WAR as if he's a semi-regular, maybe the team only plans on giving him 1/3 of that time and he should be expected to produce 0.5 WAR.

Projected playing time for the lower tier of players is not the problem. I broke out players by projected WAR. For all players, they returned 84% of projected. Looking at only the average or better projected players, the ones with 2.0 or more projected WAR, they also returned 84%. These players, with a few exceptions here and there, should have been expected to have been regulars. Since they are at least average, there should be 15 or so teams that have a worse regular at their position. I looked at players projected as stars (4.0 WAR or more). There are no exceptions here, all such players are starting pitchers or regular position players. These players returned 83% of their projected WAR.

Looking at only the 2009 signings, the players returned 86% of projected WAR, including 90% for the average and above players. Looking at 2009 signings of players I projected for 4+ WAR we run into sample size issues, it's only Mark Teixiera and CC Sabathia. They were projected for 10.6 WAR and gave the Yankees 11.1 plus another world series ring.

My projections had the 2009 group of players at a total of 132 WAR. They were paid 544 million, or 4.1 million per WAR. These are the kind of numbers that I, Tango Tiger, and other commenters on his blog are looking at. The flaw is that the players aren't actually producing that many wins. Or more accurately, we are over projecting how valuable these players are.

Although my projections are too optimistic, I wondered how they compare to the decisions made by teams in valuing players. I converted the dollars paid to a WAR projection. I divided dollars by 4.1 million to be on the same scale as the CHONE projections. Then I looked at who was closer among the 126 signings before the 2009 season. For all 126 players, I was closer to their actual 2009 WAR 72 times, the teams were right 54 times. Focusing on players where my projections disagreed with salary paid by at least 1 WAR, my estimate was closer in 9 out of 16 cases.

Looking at a guy like Matt Holliday, it does seems a little weird to see has put up 20 WAR over the last three years, I have him projected at 4.0 for 2010, and the track record of projections suggest that even that is a little bit optimistic. If you use the projections to suggest an appropriate salary, then going with the 4.1 million per WAR is fine. If you want to use Sky's 6-7 million figure based on observed dollars per win, that is fine as well, but if you're looking at how to pay a player for future production, you first need to discount the projections a bit.

If you're interested in trying to see how this works out yourself, here are the players used:

WAR WarCH War$ team player contract_aav year_sign
1.5 1.8 3.4 cha Thome, Jim 14 2003
3.8 2.2 2.7 tor Rolen, Scott 11.25 2003
2.6 1.6 2.9 hou Tejada, Miguel 12 2004
0.8 3.7 3.6 laa Guerrero, Vladimir 14.67 2004
0.8 1.5 3.2 nyn Delgado, Carlos 13 2005
1.8 2.4 3.7 det Ordonez, Magglio 15 2005
-0.4 2.8 2.7 oak Chavez, Eric 11 2005
2.4 3 3.1 sea Beltre, Adrian 12.8 2005
-0.2 3.1 2.8 stl Glaus, Troy 11.5 2005
2.9 4 4.1 nyn Beltran, Carlos 17 2005
0.4 0.7 2.6 nyn Wagner, Billy 10.75 2006
-0.6 1.1 2.3 tor Ryan, B.J. 9.4 2006
0.5 1.3 1.7 cin Hernandez, Ramon 6.9 2006
-1.6 1.5 2.2 sdn Giles, Brian 9 2006
2.1 1.8 2.3 sea Washburn, Jarrod 9.3 2006
2.4 2.1 2.9 tex Millwood, Kevin 12 2006
5.3 2.3 3.2 chn Lee, Derrek 13 2006
2.6 2.4 2.9 cha Konerko, Paul 12 2006
3 2.5 3.2 nya Damon, Johnny 13 2006
0.7 2.6 2.9 atl Hudson, Tim 11.75 2006
3.1 2.9 1.1 det Polanco, Placido 4.6 2006
2.8 4.6 2.9 atl Jones, Chipper 12 2006
-0.7 0 0.9 ari Schoeneweis, Scott 3.6 2007
-0.3 0.1 1 bal Walker, Jamie 4 2007
-0.7 0.1 2.6 mil Suppan, Jeff 10.5 2007
-0.2 0.3 0.9 chn Vizcaino, Luis 3.5 2007
0 0.4 1.1 laa Speier, Justin 4.5 2007
0 0.5 3.8 lan Schmidt, Jason 15.67 2007
0.1 0.5 0.9 tba Bradford, Chad 3.5 2007
1.8 0.6 2.1 lan Pierre, Juan 8.8 2007
0.1 0.6 2 sea Batista, Miguel 8.33 2007
-0.6 0.7 0.9 cle Dellucci, David 3.8 2007
-0.1 0.9 2.2 bos Lugo, Julio 9 2007
0.5 0.9 1.1 cin Gonzalez, Alex 4.67 2007
3.8 1 1.7 col Marquis, Jason 7 2007
0.1 1 0.2 min Redmond, Mike 1 2007
-0.8 1.1 2.4 laa Matthews, Gary 10 2007
1.8 1.1 1.3 sfn Molina, Bengie 5.33 2007
-1.2 1.3 1.6 bal Huff, Aubrey 6.67 2007
1.7 1.4 1.9 sfn Winn, Randy 7.75 2007
2.2 1.4 4.4 sfn Zito, Barry 18 2007
5.6 1.4 3.1 stl Carpenter, Chris 12.7 2007
0.9 1.7 2 bal Mora, Melvin 8.33 2007
1.7 1.8 1.1 cle DeRosa, Mark 4.33 2007
2.5 1.8 4.1 hou Lee, Carlos 16.67 2007
2 1.9 2.7 tex Padilla, Vicente 11.25 2007
4.7 2.4 3.4 bos Drew, J.D. 14 2007
3.7 2.5 2.4 chn Lilly, Ted 10 2007
0 2.6 2.3 laa Escobar, Kelvim 9.5 2007
-0.7 2.9 4.1 chn Soriano, Alfonso 17 2007
2.4 2.9 1.3 was Johnson, Nick 5.5 2007
2.5 3.2 3.7 chn Ramirez, Aramis 15 2007
1.5 4 2.7 kca Meche, Gil 11 2007
0.7 4.6 3.2 bos Ortiz, David 13 2007
0 0 1.1 mil Riske, David 4.33 2008
-0.3 0.1 1 phi Romero, J.C. 4 2008
0.9 0.1 0.6 sln Franklin, Ryan 2.5 2008
-0.3 0.1 1 tba Percival, Troy 4 2008
0 0.3 0.3 nyn Anderson, Marlon 1.1 2008
0.2 0.4 1.2 cha Linebrink, Scott 4.75 2008
-0.3 0.4 1 kca Mahay, Ron 4 2008
1.2 0.4 1.1 mil Kendall, Jason 4.6 2008
0.1 0.4 0.5 nya Molina, Jose 2 2008
0 0.4 0.4 phi Stairs, Matt 1.6 2008
0 0.4 0.5 tor McDonald, John 1.9 2008
0.8 0.6 0.9 col Torrealba, Yorvit 3.6 2008
4.8 0.6 1.6 sln Pineiro, Joel 6.5 2008
0.8 0.7 1.3 cha Dotel, Octavio 5.5 2008
0 0.7 1.2 was Young, Dmitri 5 2008
0.1 0.9 3.4 nyn Sheffield, Gary 14 2008
1.3 0.9 0.4 was Belliard, Ronnie 1.75 2008
0.5 1 2.4 ari Byrnes, Eric 10 2008
2.5 1 1.5 cha Pierzynski, A.J. 6.1 2008
1.5 1 2.8 cin Cordero, Francisco 11.5 2008
-2 1 2.9 kca Guillen, Jose 12 2008
1.6 1 1.5 nyn Castillo, Luis 6.25 2008
1.9 1 2.9 sfn Rowand, Aaron 12 2008
-0.1 1.2 2.9 sea Silva, Carlos 12 2008
1.3 1.3 1 phi Feliz, Pedro 4.25 2008
0.6 1.4 0.6 nyn Castro, Ramon 2.3 2008
-0.3 1.5 2.7 cha Dye, Jermaine 11 2008
0.8 1.6 4.4 tex Jones, Andruw 18.1 2008
0 1.7 2.7 cle Westbrook, Jake 11 2008
4.3 1.8 2 mil Cameron, Mike 8 2008
1.9 1.8 2.9 min Nathan, Joe 11.75 2008
3.8 2.2 4.4 laa Hunter, Torii 18 2008
2 2.2 3.7 nya Rivera, Mariano 15 2008
5.1 2.4 4.4 sea Suzuki, Ichiro 18 2008
1.2 2.5 3 bos Lowell, Mike 12.5 2008
4 2.6 3.2 nya Posada, Jorge 13.1 2008
-0.1 2.6 4.4 tor Wells, Vernon 18 2008
0.1 2.8 2.9 det Guillen, Carlos 12 2008
3.6 2.9 4.5 chn Zambrano, Carlos 18.3 2008
3.4 3.6 3.4 cha Buehrle, Mark 14 2008
6.6 4.6 2.8 atl Vazquez, Javier 11.5 2008
2.8 4.8 5.6 nyn Santana, Johan 23 2008
7.3 5 3.3 tor Halladay, Roy 13.33 2008
4.4 5.6 6.7 nya Rodriguez, Alex 27.5 2008
0 0 0.2 atl Glavine, Tom 1 2009
-0.8 0 1 cin Weathers, David 3.9 2009
-0.4 0 0.2 tex Guardado, Eddie 1 2009
0.9 0 0.6 hou Moehler, Brian 2.3 2009
0 0 0.5 phi Eyre, Scott 2 2009
1.5 0 0.6 phi Park, Chan Ho 2.5 2009
0 0 0.2 tba Isringhausen, Jason 0.75 2009
-0.1 0 0.6 lan Mota, Guillermo 2.35 2009
0.1 0 0.2 col Rusch, Glendon 0.75 2009
0 0 0.1 ari Gordon, Tom 0.5 2009
0.4 0 0.2 free Ortiz, Russ 0.75 2009
-0.5 0.1 0.6 hou Brocail, Doug 2.5 2009
-0.3 0.1 0.2 hou Boone, Aaron 0.75 2009
-0.1 0.1 0.4 tba Shouse, Brian 1.55 2009
1 0.1 0.5 cin Rhodes, Arthur 2 2009
-0.1 0.2 0.5 col Embree, Alan 2.25 2009
0.9 0.2 0.5 hou Hampton, Mike 2 2009
0 0.2 0.4 kca Ramirez, Horacio 1.8 2009
0.6 0.2 0.2 lan Ausmus, Brad 1 2009
1.5 0.2 1.5 mil Hoffman, Trevor 6 2009
0.5 0.2 0.2 nya Moeller, Chad 0.8 2009
-1 0.2 0.5 cin Lincoln, Mike 2 2009
0 0.2 0.2 det Rincon, Juan 0.75 2009
0.1 0.2 0.1 free Tavarez, Julian 0.5 2009
0.1 0.2 0.2 kca Wright, Jamey 0.8 2009
0.5 0.3 0.4 bal Hendrickson, Mark 1.5 2009
-0.3 0.3 0.4 bos Kotsay, Mark 1.5 2009
-0.3 0.3 0.2 mil Julio, Jorge 0.95 2009
0.4 0.3 0.8 oak Springer, Russ 3.3 2009
-0.6 0.3 0.2 tor Millar, Kevin 0.85 2009
0.3 0.3 0.5 free Beimel, Joe 2 2009
-0.1 0.4 0.2 ari Clark, Tony 0.8 2009
-0.3 0.4 0.2 tex Benson, Kris 0.75 2009
-0.9 0.4 0.4 hou Erstad, Darin 1.75 2009
0.3 0.4 0.9 hou Hawkins, LaTroy 3.5 2009
0.7 0.4 1.1 kca Farnsworth, Kyle 4.6 2009
0.2 0.4 0.1 phi Cairo, Miguel 0.5 2009
-0.3 0.4 0.2 pit Monroe, Craig 0.75 2009
0 0.4 0.7 sfn Howry, Bobby 2.75 2009
0.3 0.4 0.2 sln LaRue, Jason 0.95 2009
-0.5 0.4 0.4 free Ohman, Will 1.55 2009
0.5 0.4 0.5 sln Miller, Trever 2 2009
3.7 0.5 0.4 cle Pavano, Carl 1.5 2009
1.5 0.5 0.9 laa Oliver, Darren 3.67 2009
1.2 0.5 0.2 sdn Blanco, Henry 0.75 2009
0.2 0.5 0.5 sea Griffey, Ken 2 2009
0.8 0.5 1 sfn Affeldt, Jeremy 4 2009
1.2 0.5 0.2 tex Vizquel, Omar 1 2009
0.2 0.5 0.4 free Reyes, Dennys 1.5 2009
-0.9 0.6 0.2 atl Norton, Greg 0.8 2009
0.7 0.6 1 det Lyon, Brandon 4.25 2009
0.5 0.6 0.2 flo Helms, Wes 0.95 2009
-0.1 0.6 0.4 kca Bloomquist, Willie 1.55 2009
-0.1 0.6 1 nya Marte, Damaso 4 2009
0 0.6 0.5 nyn Cora, Alex 2 2009
0.6 0.6 1.6 phi Moyer, Jamie 6.5 2009
0 0.6 0.5 pit Vazquez, Ramon 2 2009
-1 0.7 0.6 atl Anderson, Garret 2.5 2009
0.6 0.7 0.4 hou Blum, Geoff 1.5 2009
0.2 0.8 0.2 cha Colon, Bartolo 1 2009
-0.3 0.8 0.2 hou Michaels, Jason 0.75 2009
-0.2 0.8 0.3 lan Loretta, Mark 1.25 2009
-0.7 0.8 3 phi Lidge, Brad 12.5 2009
-0.5 0.8 0.2 sfn Aurilia, Rich 1 2009
1.1 0.8 0.2 tba Kapler, Gabe 1 2009
0.2 0.8 0.2 tex Jennings, Jason 0.8 2009
-0.1 0.8 0.1 tor Barrett, Michael 0.6 2009
1.3 0.9 0.6 bal Izturis, Cesar 2.5 2009
2.8 0.9 0.2 mil Counsell, Craig 1 2009
0.2 0.9 0.1 sea Sweeney, Mike 0.5 2009
2.9 0.9 0.2 sfn Uribe, Juan 1 2009
1 1 0.5 cin Hairston, Jerry 2 2009
0.8 1 0.4 hou Rodriguez, Ivan 1.5 2009
-0.1 1 0.1 lan Mientkiewicz, Doug 0.55 2009
-0.9 1 1.2 sln Looper, Braden 4.75 2009
1.7 1.1 0.4 atl Ross, David 1.5 2009
1.8 1.1 0.5 bal Zaun, Gregg 2 2009
0 1.1 0.7 kca Cruz, Juan 3 2009
3.4 1.1 1 laa Rivera, Juan 4.25 2009
-0.3 1.1 0.2 sdn Floyd, Cliff 0.75 2009
1.3 1.2 1.2 bos Varitek, Jason 5 2009
0.4 1.2 2.5 cle Wood, Kerry 10.25 2009
0.9 1.2 0.2 det Everett, Adam 1 2009
1.3 1.2 1 min Punto, Nick 4.25 2009
1.5 1.2 0.4 nyn Tatis, Fernando 1.7 2009
-0.3 1.2 0.2 oak Garciaparra, Nomar 1 2009
0.8 1.2 0.4 pit Hinske, Eric 1.5 2009
4.6 1.3 0.9 ari Lopez, Felipe 3.5 2009
2.5 1.3 1.2 bos Penny, Brad 5 2009
0.4 1.3 2.1 laa Fuentes, Brian 8.75 2009
3 1.3 1.2 lan Wolf, Randy 5 2009
0.7 1.3 0.2 sdn Eckstein, David 0.85 2009
0.8 1.3 0.5 tor Barajas, Rod 1.85 2009
1.9 1.4 1 bos Wakefield, Tim 4 2009
4.2 1.4 2.6 phi Ibanez, Raul 10.5 2009
2.8 1.5 0.3 sea Branyan, Russell 1.4 2009
0.3 1.5 2.3 sfn Renteria, Edgar 9.25 2009
0.6 1.6 1 oak Cabrera, Orlando 4 2009
0.9 1.6 2 was Guzman, Cristian 8 2009
-0.6 1.7 2 tba Burrell, Pat 8 2009
2.9 1.8 1.8 lan Hudson, Orlando 7.3 2009
0.8 1.8 2.5 sln Lohse, Kyle 10.25 2009
2.4 1.9 1.8 ari Garland, Jon 7.25 2009
1.5 1.9 1.3 bos Smoltz, John 5.5 2009
4.2 1.9 1.4 lan Blake, Casey 5.8 2009
1.9 1.9 0.6 min Crede, Joe 2.5 2009
-0.8 1.9 2.9 nyn Perez, Oliver 12 2009
-0.2 1.9 1.3 oak Giambi, Jason 5.25 2009
0.4 1.9 2 sfn Johnson, Randy 8 2009
2.5 2 1.2 laa Abreu, Bobby 5 2009
1.8 2.1 3 cin Arroyo, Bronson 12.25 2009
0.3 2.1 3 nyn Rodriguez, Francisco 12.33 2009
3.6 2.2 3.2 chn Dempster, Ryan 13 2009
1.9 2.2 2.4 col Cook, Aaron 10 2009
1.2 2.2 2.4 was Dunn, Adam 10 2009
3.3 2.4 2.6 nya Pettitte, Andy 10.5 2009
3.9 2.5 3.9 tex Young, Michael 16 2009
1.2 2.6 3.5 cle Hafner, Travis 14.25 2009
1.2 2.7 1.3 oak Ellis, Mark 5.5 2009
3.2 2.8 2.4 lan Furcal, Rafael 10 2009
1 3.1 2.4 chn Bradley, Milton 10 2009
2.7 3.3 5.5 lan Ramirez, Manny 22.5 2009
2.7 3.6 3.7 atl Lowe, Derek 15 2009
3.1 3.9 4 nya Burnett, A.J. 16.5 2009
5.1 4.7 5.5 nya Teixiera, Mark 22.5 2009
6 5.9 5.6 nya Sabathia, CC 23 2009