Now, one of the unique things I don't do with my Elo ratings is reset or adjust them at the end of each season. I see (for better or worse) an Elo rating as a self-correcting system, so if a team's form significantly changes because of offseason moves, managerial changes, etc., the model will correct their rating based on new results. This has a very interesting effect on the Monte Carlo predictions. In fact, one could go so far as to say it breaks the Monte Carlo predictions - skewing them so hard to the last year's form that they are hardly meaningful. Let's take a look at the Monte Carlo estimations to see what I mean:
These are the estimations for year end 2014 before a single game has been played. Right now Chivas USA is predicted to have no chance of making the playoffs and DC United has about a 1/10,000 chance. Perhaps the most disturbing prediction here is Toronto; even after acquiring Michael Bradley and Jermaine Defoe (which this model has no way of taking into consideration) they have less than a 1% chance to make the playoffs. On the top of the table, the 2013 MLS Cup finalists Real Salt Lake and Sporting Kansas City are virtually guaranteed playoff spots.
There are positives and negatives to this. On one hand, MLS is a league of parity and we've seen many teams go from bottom dwellers to cup contenders in a single season (or vice versa). On the other hand, does anyone really expect Chivas USA to compete for a playoff spot this year? Even the odds makers have them at 1/66 chance of winning MLS Cup before a single game has been played.
My Monte Carlo numbers are an extreme. What they are doing is essentially inserting a carbon copy of the teams we witnessed last year and playing out an entire season between those carbon copies. Sports Club Stats' numbers are the polar opposite. They are erasing all history and assigning every team a 50% or 55.6% chance - based on conference - of making the playoffs. Reality is probably somewhere in between. It's hard to claim that Chivas USA has a 55.6% chance of making the playoffs in 2014. In fact, when you compare the two predictions, 55.6% or 0%, most people would probably agree that 0% is the more likely number.
But at the same time, Chivas is a special example. Even talking about the two other teams that had disastrous 2013 seasons - Toronto and DC - my predictions seem low. A lot of that has to do with Toronto and DC's offseason revamping. But, it's not a stretch to assume that teams coming off a disastrous season are going to make a lot of changes, and that's something this model doesn't assume. Like my Elo ratings themselves, these numbers will only reflect the change in form after a few 2014 games have been played.
Personally, I like to look at these along side Sport Clubs Stats' playoff chances and use the two as a range. I do think that using a form rating is a better way to run Monte Carlo estimations than league average home win percentages, but form rating is in itself a subjective (at least partially) measure. So, take these with a grain of salt, if you take them at all. But towards the end of the season when form is being determined by 2014 results - that's when this method shines.
