+ Show Spoiler +
Leagues can be an unhelpful "I suck" banner: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#5
Scaling MMR-1.96*sigma so that losses don't have a greater impact than wins: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#10
On dealing with new players and season resets within the proposed ladder system: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#12
Negative psychology caused by displaying "favored": http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#15
Loss aversion and improving the division system: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449¤tpage=2#33
Giving extra points for map of the week or other "events" is a bad idea because it distorts ranks: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449¤tpage=3#47
Currently, players can camp on Master's league by playing only a few games each season: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449¤tpage=3#58
Scaling MMR-1.96*sigma so that losses don't have a greater impact than wins: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#10
On dealing with new players and season resets within the proposed ladder system: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#12
Negative psychology caused by displaying "favored": http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449#15
Loss aversion and improving the division system: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449¤tpage=2#33
Giving extra points for map of the week or other "events" is a bad idea because it distorts ranks: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449¤tpage=3#47
Currently, players can camp on Master's league by playing only a few games each season: http://www.teamliquid.net/forum/viewmessage.php?topic_id=389449¤tpage=3#58
Following from the success of my Battle.net feedback thread last week, I’ve compiled my thoughts on the ladder system. About half of the arguments should be familiar if you’ve read my previous posts. I outline the flaws of the bonus pool, points and division systems. I suggest how they can be fixed and integrated with a global ladder. This thread can be found on the Battle.net forums here.
The square brackets in the summary indicate the section where additional information can be found.
1. Summary
The points system is flawed because of the bonus pool. The bonus pool distorts points, making it inaccurate for ranking players [2.2]. The goals of the system appear to be: (1) to prevent stagnation, (2) to encourage playing, (3) a positive psychology version of a decay system to allow activity to be factored into points and ranks.
But it fails at these goals. On the psychology of (1) and (2) the bonus pool causes the inflation of other player’s points, which in turn perpetually causes your rank to fall, a punishment symmetric to the reward of increasing points. So overall, it's not a psychological reward, it creates a “treadmill effect” [2.1, 2.3]. On (3), For the purpose of accurate ranking, there is no reason to account for activity, other than as a proxy to uncertainty about MMR. But there is no need to use activity as that proxy because uncertainty about MMR is an already-known quantity which can be included into points [2.4].
To address (2), a wider range of shinier rewards and increased grind should be added to the leveling system [3.1]. Similar to TrueSkill, points should be redesign to be MMR-1.96*sigma, where sigma is the uncertainty about MMR that is increased at the end of the week if the player isn’t sufficiently active. This addresses (1) by explicitly accounting for uncertainty, so there is no need to use activity measured through bonus pool as a proxy [3.2].
The only remaining legitimate role of bonus pool is (3). The following changes are required to properly address stagnation while avoiding the flaws of the current bonus pool system: Points should be changed to converge to MMR-1.96*sigma, instead of MMR. To reduce distortions, the amount of bonus pool can then be significantly reduced, and the more bonus pool a player has the faster it should be consumed. These changes are only sensible in light of the suggestions to address (1) which accounts for psychology, and (2) which accounts for accuracy. To further reduce distortions and the treadmill effect, bonus pool should only be rewarded in bulk at the start of the week and the ladder should only be updated then, instead of in real time [3.3].
The most fundamental purpose of a ladder system is to correctly and accurately ranked players. Since division ranks are meaningless [4.1], a global ladder needs to be implemented using the changes to the points and bonus pool system described above. For each player, the global ladder should give a percentile out of all active players [4.2]. GM league is flawed and should be scrapped. It would also be obsolete if there’s a global ladder [4.4].
As Blizzard says, unranked play should largely address ladder anxiety. However, the following changes should help to further reduce ladder anxiety. Ranks should be made less prominent and points more prominent in order to reduce the treadmill effect [5.1]. A tutorial about ranked games should be produced to explain that the design of the ladder system, the fact that MMR is self-correcting, implies that it is impossible to stuff up your current season stats no matter how long the losing streak [5.2]. Past season history should be hidable as it is the only remaining way to permanently stuff up your account [5.3]. Team games should be encouraged before solo games [5.4].
2. The Flaws of the Points and Bonus Pool System
In this section, I will explain how the bonus pool system fails as a positive psychological gimmick and distorts points and ranks.
2.1 Bonus Pool is not a catch up mechanism
Blizzard claims that the bonus pool is to help casuals keep up on the ladder.
Q. What is the Bonus Pool and how are bonus points acquired?
A. The Bonus Pool is an accumulation of points that every player receives whether they're online and playing or not. They're essentially used as a means to help give a player a catch-up boost if they haven't played in a while. The pool does have a cap, but it increases slowly until the end of a season.
Source: http://us.battle.net/sc2/en/blog/110519
A. The Bonus Pool is an accumulation of points that every player receives whether they're online and playing or not. They're essentially used as a means to help give a player a catch-up boost if they haven't played in a while. The pool does have a cap, but it increases slowly until the end of a season.
Source: http://us.battle.net/sc2/en/blog/110519
This is completely wrong. The bonus pool perpetually inflates the points of more active players, which causes your rank to fall and continually requires you to play on a daily basis, even to maintain the same position. This creates a "treadmill effect”. If Alice and Bob are equally skilled and Alice becomes less active, then her points will diverge from Bob's. The bonus pool doesn't help Alice close this divergence, it’s the cause of this divergence in the first place. It’s the problem, not the solution.
If there were no bonus pool, after approximately 30 games, players will hit their MMR and fluctuate around this point unless there’s a legitimate change in their skill. This is what happens when bonus pool is used up anyway. In this case, players would stagnate at their true skill level, meaning that there would be no need to catch up. Thus, having no bonus pool system would be far more helpful to players catching up, since they won’t have to chase a moving target. So Blizzard is wrong about bonus pool and their justification for it is completely fallacious.
2.2 Bonus Pool distorts ranks and increases the time taken for points to self-correct
The bonus pool system causes points to inflate until the season lock, so that the ladder never really stabilizes. Suppose that Alice and Bob have used up their bonus pool and are ranked 10 and 15 respectively. Alice is more skilled than Bob. The next day, Alice doesn’t play, so she falls to rank 14 because other players have used their bonus pool. Bob uses his bonus pool moving him to rank 12. Bob is now erroneously ranked higher than Alice, until Alice and the other players use their bonus pool to increase their points and bump Bob down. Until the season lock, this situation is perpetual.
The bonus pool also obstructs and hinders MMR from self-correcting, because a player’s points cannot decrease until bonus pool is used, even when skill legitimately does. For example, suppose Alice plays actively, she has true MMR 1000, and bonus pool is given at the rate of 50 per week. In week 5, Alice will correctly have 1250 points. Now suppose that she takes a week off and her true MMR has dropped to 800 (e.g. she had a real life injury, or forgot how to play, etc.). Then in week 6, her correct points would be 1100 (1300 minus 200 MMR for loss skill). But, she’ll still be stuck on 1250 points, which wouldn’t decrease until the bonus pool is used up. Even without the HotS change where lost points are absorbed by the bonus pool, Alice's points will on average change very little, until the bonus pool is spent. Hence, Alice's points have been distorted to be erroneously higher than is correct, with adjustment only happening after the bonus pool is used up. In contrast, adjustment would be immediate had there not been a bonus pool system.
These distortions mess up ladder ranks.
2.3 Bonus pool is not a “feel good” decay system, nor a psychological reward
Some apologists of the bonus pool system claim that it’s all about positive psychology. Bonus pool prevents stagnation by letting points increase even if skill and MMR plateaus. A decay system is defined as one that deducts points at the end of each week where the player has not played enough games. Instead of a decay system where players are punished for not playing, the bonus pool system "rewards" players for playing.
At least that's what it tries to be. Bonus pool was seemingly designed with the same philosophy as WoW’s rested XP system. Back in WoW's beta, instead of punishing players by reducing 50% of XP gain when they've played too much, Blizzard doubled all XP and made rested XP a psychological reward by having it always give a 100% bonus.
But such logic cannot be applied to a ranking system, where one player's gain is another player’s loss. Every day you log in, you see your rank fall because of the treadmill effect. Accounting for the fact that other player’s bonus pool causes your rank to continually fall, obviously the reward of increased points is symmetric to the punishment of falling ranks, it’s self-defeating, it cancels itself out.
Thus, these positive psychology arguments are also completely wrong. However, the bonus pool system has replaced the traditional decay system. In this section, I’ve shown why bonus pool is a flawed decay system for the purposes of positive psychology. In 2.4, I show why it fails as a decay system for the purposes of accurate ranking.
2.4 Bonus pool rewards activity in a needless and suboptimal way
Another common argument is that the bonus pool allows for the ladder to reward activity without rewarding mass gaming. While it’s a good idea to encourage activity for the sake of getting people to play the game, this should be done with a levelling system, not a bonus pool system, because the latter distorts ranks as explained in 2.2.
The bonus pool tends to increase the points of active players. But for the purpose of accurate ranking, why should activity even matter?
If we were psychic and simply knew the skill of each player at a given moment, without needing any games to be played, then we would only use this knowledge for ranking, i.e. in an ideal world ranking will be 100% skill based. However, we don't completely know someone's skill at a given moment, unless they play. This is the only reason to consider factoring activity into points and ranks, as higher activity is usually a good proxy to a higher probability that the player's MMR is correct. To the extent that we have good knowledge of a player's current skill, activity should not matter for the purpose of ranking.
This means that ideally, we want to minimize the weight given to activity as a factor, subject to the constraint that the player is active enough to give a reasonably good estimate of his current skill. For example, to have accurate ranks, decay systems that penalize players after a week of inactivity are superior to the current bonus pool system, because they reduce the weight given to activity as it doesn't matter as long as you play a little each week. The bonus pool system, however, requires that you be active always, every single day, so does not satisfy the above-mentioned criteria. Therefore, decay systems result in significantly less distortions than bonus pool systems, particularly for active players.
Suppose it takes about 30 games for the ladder system to calculate a player’s MMR to within an acceptably small uncertainty of 50, any more games would just cause uncertainty to fluctuate a little around 50. Alice joins the ladder on Jan 6. By Feb 20 she has played 90 games, just enough to consume her bonus pool, giving her 1500 MMR with an uncertainty of 50. Bob joins the ladder on Feb 20 and plays 30 games that day, ending with 1500 MMR and the uncertainty about his MMR would also be 50. Bob will have fewer points than Alice because he is 60 games short of consuming his bonus pool. But for the purposes of accurate ranking, there is absolutely no reason why Alice should have more points, since they both have equal MMR and equal uncertainty about MMR. Note that a decay system does not face this problem. Therefore, activity as measured by consumed bonus pool can be a bad proxy to uncertainty about MMR.
In fact, it’s completely unnecessary to use activity as this proxy, because the system already measures it directly and it’s called sigma. Therefore, bonus pool is flawed because it factors activity into points and ranks, when there is no reason for activity to matter since what we ultimately want from it is uncertainty about the player’s MMR, which is a number the system already knows.
3. Fixing the Points and Bonus Pool System
Above we have identified 4 goals of bonus pool.
(1) As a catch up mechanism.
(2) To prevent points from stagnating.
(3) To encourage playing more games.
(4) To allow activity to be factored into points and ranks.
It is logically impossible to achieve (1) for any serious ranking system, as explained in 2.1, so this goal will be ignored. So far I have shown that the bonus pool fails at all of these goals, except (2). But worse than failing, I have shown that bonus pool distorts points and ranks thereby screwing up the ladder. In this section, I suggest how to design a ladder system that achieves all of these goals, while only distorting points and ranks to the smallest possible extent.
3.1 Only encourage playing and reward activity through the leveling system.
Cosmetic rewards should be used to address (3) because any encouragement through giving bonus points is self-defeating. These cosmetic rewards should be attached to the leveling system. They should include portraits, decals, unit models or B.net backgrounds, and not be as completely lame as they currently are in the HotS beta. There would need to be an option to turn off special decals and unit models as explained in section 40 in the Battle.net feedback thread. In addition, the leveling system should be made more grindy. For example, playing games could reward tokens that are used to purchase the shiniest cosmetic rewards. The goal here is purely to encourage playing.
3.2 Make a more accurate ranking system by ignoring activity and explicitly including sigma
Note that goals (2) and (3) already address psychology. Therefore, in addressing (4), we are purely concerned with accurate ranking. To explicitly account for uncertainty about MMR, instead of indirectly using consumed bonus pool as an imperfect proxy, points should simply be set to MMR-1.96*sigma (possibly scaled so that the numbers fall into a reasonable range), which is the lower bound of the 95% confidence interval for MMR. Then the current bonus pool system becomes unnecessary. This is essentially what is done in TrueSkill (it uses 3 instead of 1.96).
Here sigma is the uncertainty about MMR, which is initially chosen so that points are equal to 0 for new accounts. It should increase at the end of each week if the player has not played enough games, reflecting the fact that we are less certain about a player’s current skill if he has not recently played.
Currently, the ladder system ranks by points that converge to MMR as long as the player is active enough to spend bonus pool. The proposed system converges to MMR as long as the system gets increasingly sure about the player’s MMR.
A technical issue:
+ Show Spoiler +
There is a potential complication with setting points to MMR-1.96*sigma. In SC2's ladder system, if you win against a higher skilled player, your MMR will increase, but so will sigma, because the result is a surprise. So despite winning against the odds, MMR-1.96*sigma can change little or even decrease when you beat a higher skilled player.
This is caused by a combination of 2 issues. Firstly, in SC2's ladder system sigma doesn't necessarily decrease as it does in TrueSkill (so this problem is nonexistent in TrueSkill). Secondly, in both systems skill is modelled by a symmetric normal distribution. This means that if you a win against a higher skilled player, the SC2 ladder system says that your skill level is higher than previously believed, but the surprise of your improbable win makes the spread of your likely skill level wider, both above and below what was previously believed. However, this is clearly wrong, the spread of your likely skill level should be heavily weighted above what was previously believed and lightly weighted below what was previously believed, instead of being widely spread in both directions.
For example, recall that at SC2's launch there were a few players with records around 50-0 (Ownage and CauthonLuck) who failed to be promoted. They clearly passed the MMR threshold, but failed the sigma threshold. But after winning 50-0, the system should clearly have a lower bound on their skill level, and hence they should have been promoted. This problem demonstrates an ineluctable mathematical truth about normal distributions, sigma basically measures the spread above and below the mean and both must be equal. And we know from the Bayesian inference formula shown at Blizzcon that Blizzard uses normal distributions.
To fix this, skill needs to be modeled with a skewed, not symmetric distribution, like a gamma or skew-normal distribution. Due to the lack of symmetry, MMR-1.96*sigma would need to be replaced with the more general concept of the 2.5th quantile of the posterior distribution for skill (in the case of a normal distribution, both concepts are completely identical). This would solve both problems outlined here. All of my suggestions are still applicable in this case. The change to a skewed distribution should be adopted even if points aren’t changed to MMR-1.96*sigma, because it’s more correct and fixes the promotion problem outlined above.
This is caused by a combination of 2 issues. Firstly, in SC2's ladder system sigma doesn't necessarily decrease as it does in TrueSkill (so this problem is nonexistent in TrueSkill). Secondly, in both systems skill is modelled by a symmetric normal distribution. This means that if you a win against a higher skilled player, the SC2 ladder system says that your skill level is higher than previously believed, but the surprise of your improbable win makes the spread of your likely skill level wider, both above and below what was previously believed. However, this is clearly wrong, the spread of your likely skill level should be heavily weighted above what was previously believed and lightly weighted below what was previously believed, instead of being widely spread in both directions.
For example, recall that at SC2's launch there were a few players with records around 50-0 (Ownage and CauthonLuck) who failed to be promoted. They clearly passed the MMR threshold, but failed the sigma threshold. But after winning 50-0, the system should clearly have a lower bound on their skill level, and hence they should have been promoted. This problem demonstrates an ineluctable mathematical truth about normal distributions, sigma basically measures the spread above and below the mean and both must be equal. And we know from the Bayesian inference formula shown at Blizzcon that Blizzard uses normal distributions.
To fix this, skill needs to be modeled with a skewed, not symmetric distribution, like a gamma or skew-normal distribution. Due to the lack of symmetry, MMR-1.96*sigma would need to be replaced with the more general concept of the 2.5th quantile of the posterior distribution for skill (in the case of a normal distribution, both concepts are completely identical). This would solve both problems outlined here. All of my suggestions are still applicable in this case. The change to a skewed distribution should be adopted even if points aren’t changed to MMR-1.96*sigma, because it’s more correct and fixes the promotion problem outlined above.
3.3 Do the bonus pool correctly: make points converge to MMR-1.96*sigma significantly reduce bonus pool, make bonus pool consume at a faster rate the more bonus pool a player has, give out bonus pool weekly not hourly, update the ladder weekly not in real time, deemphasize ranks and emphasize points
So far we have addressed (3) and (4) without needing the distortionary bonus pool system. The only way to address stagnation without some sort of bonus pool is to increase every player’s points every hour, regardless of their activity. This is not a completely terrible idea. However, this section explains how bonus pool can be redesigned to address (2), while minimizing the distortionary and treadmill effects that are caused by the current system.
The only remaining legitimate reason for bonus pool is to prevent stagnation. Firstly, the suggestion in 3.2 should be slightly amended so that points converge to MMR-1.96*sigma instead of being precisely that as having both a decay system and bonus pool system doesn’t make sense. Next, bonus pool can be significantly reduce, from about 110 per week in HotS, to 20 per week. Additionally, the more bonus pool a player has the faster it should be consume. For example, if you have 100 bonus pool, getting 12 points for winning should use, say, 24 bonus pool, if you have 200 bonus pool, it should use 84 bonus pool. Note that these changes only make sense when implemented together with the suggestions in 3.1 and 3.2 that have already addressed the need to encourage activity and account for uncertainty about MMR. Hence, these changes to trivialize bonus pool have only the purpose of preventing stagnation and nothing more.
These are positive changes because significantly reducing the bonus pool would significantly reduce the distortionary and treadmill effects it creates. Allowing bonus pool to be consumed faster when players have large bonus pools partly addresses the problems in the second example in 2.2 and the “Jan 3 vs Feb 20” example in 2.4. It also partly addresses (1), but no changes in any serious ranking system can (nor should) entirely fix (1).
In addition, bonus pool should be given in bulk, once weekly, instead of in small amounts each hour, and the ladder should only be updated at this time, instead of in real time. Updating the ladder once a week will significantly reduce the treadmill effects since players will no longer see their rank perpetually fall due to other player’s bonus pools. But more importantly, these changes will mostly eliminate the distortionary effects that bonus pool has on ranks as explained in the first example of 2.2. In that example, Alice is more skilled than Bob. She doesn't play for a day and falls below Bob's rank as a result of Bob's bonus pool. So the ladder ranks have become wrong. Now if the ladder were to update only once weekly and bonus pool were changed as I've suggested, then Alice would be able to get back ahead of Bob, before the next ladder snapshot. If she didn't, it would be because she was inactive for the week, so it could be justified that her rank should fall as a small penalty for the chance that her skill has decreased due to prolonged inactivity. However, such an argument cannot be applied to the current bonus pool system because Alice would not lose any skill due to not having played for one hour or one day. The skill lost for 2 weeks of inactivity is far more than 14 times the skill lost in 1 day of inactivity.
Lastly, to further reduce the treadmill effect and to maximize the benefits of having prevented points from stagnating, ranks need to be removed from the matchmaking page and the score screen. Instead, points should be emphasized, as they no longer stagnate. Ranks should be kept in the ladder page in the profile. They are critically important for competition in a competitive game.
4. The Meaningless Division System vs a Global Ladder
4.1 Division ranks have no useful interpretation
The division system is pointless. Your division rank says nothing about how you compare with other players, even within the same league.
Blizzard claims that division ranks have a meaning.
[C]limbing to (for example) Rank 2 Diamond will mean that you are in the top 2% of all Diamond players, and you are very close to moving into the Master League. Similarly, Rank 50 Platinum is in the top 50% in the Platinum league, and so forth.
Source: http://us.battle.net/sc2/en/blog/7157308/Season_8_Now_Locked_and_Big_Changes_Coming_Next_Season-9_6_2012
Source: http://us.battle.net/sc2/en/blog/7157308/Season_8_Now_Locked_and_Big_Changes_Coming_Next_Season-9_6_2012
But this is simply untrue. If players in the same league were randomly placed into divisions, and if for each player in the league, their skill doesn’t change throughout the season, then it would only be true that being rank 2 will on average mean that the player is in the top 2%. Even in this ideal case, there is no guarantee that some randomly created divisions aren’t by chance more skilled than others. But more importantly, this isn’t how divisions are created. They’re created on a first come, first serve basis, and hence divisions created at the start of the season will tend to have more active, and hence more skilled, players than divisions created later.
Moreover, with the change that allows players to leave leagues, players can shop for easier divisions by checking the points of players in it using SC2ranks. This actually gives empirical evidence that Blizzard’s statement is factually wrong, so it is untrue that removal of division tiers somehow validates division ranks. There is no point in ranking against 100 arbitrary and faceless players. Despite Rob Pardo’s misguided attempt to convince us otherwise, there is still no reason to care.
Leagues do not solve the problem of division ranks being meaningless and there being no way to get a reasonable measure of your skill relative to all active players. The 5 leagues other than Masters and GM cover an approximately 20% skill range, in the sense that Platinum league contains players in the top 20%-40%. This is a very large skill gap.
4.2 Creating a global ladder
The solution is to create a global ladder, where all players in the season and on the server are ranked by points. This does not necessarily mean that the division ladders should be removed, just that division ranks are meaningless. Both global and division ladders can coexist.
The points system suggested in 3.2 and 3.3 should be used to ranked players on a global ladder. As previously explained, if the bonus pool is not scrapped, then the global ladder should be updated once weekly instead of in real time. A percentile should be used. For example, 84.2 instead of 3756 out of 20000. This number is very easy to understand, 84.2 simply means that you’re better than 84.2% of all players that are ranked (this is exactly how university admission ranks work in Australia).
Attrition throughout the current season can reduce the usefulness of the global ladder. For example, if you're in the top 30% only because 50% of players have 0-1 records and are no longer active, then you're not really in the top 30%. It isn't helpful to be compared with people who no longer play. Therefore, the default view of the global ladder and the percentile should exclude inactive players, but there should also be a view that includes all players in the current season. Sometimes you want to include marathon runners who failed to reach the finish line, but other times, only competition with marathon runners who cross the finish line matters. Alternatively, there could be a minimum number of games per season, say 10, before a player is added to the global ladder, any less means that the player isn’t serious enough to be ranked.
4.3 Tired arguments against a global ladder
Objections to a global ladder can generally be classified into 2 types of arguments. Firstly, the global ladder causes ladder anxiety and hurts casuals. Secondly, players cannot be accurately ranked on a global ladder so that Blizzard shouldn’t bother.
On the first argument, section 5 outlines changes to reduce ladder anxiety. In fact, the ladder system I’ve proposed would be far more accommodative to casuals, as it fixes the treadmill effect of the current system, removes the obsessive, harmful and meaningless fixation on division ranks, prevents points from stagnating, and encourages playing, while at the same time making the ladder far more accurate. But ultimately, this is a competitive game. Competition and epeen matters. The fundamental purpose of a ladder system is to correctly and accurately rank players, not to tend to the hurt feelings of sensitive players or casuals.
When Blizzard introduced win loss stats in HotS, they said that:
Players who play competitively on the ladder can now better track their progress regardless of which league they’re in. And those players that would like to enjoy the benefits of matchmaking, but are not interested in the pressure of being ranked can now use the unranked play mode.
Source: http://us.battle.net/sc2/en/blog/7634957/
Source: http://us.battle.net/sc2/en/blog/7634957/
This argument can be applied to a global ladder. After all, a global rank is just another stat.
Ironically, the “bad for casuals” argument is seemingly most often made by players who strongly defend Blizzard’s current ladder system as being good for casuals despite either not realizing or ignoring the fact that bonus pool hurts casuals due to the treadmill effect, that the removal of division tiers makes it harder for casuals to rank up on the division ladder (this was the most significant change Blizzard has made to improve accuracy and comparability in the ladder system), and that Blizzard brands a mediocrity badge called “Bronze League” on 42.1% of players forehead. They defended Blizzard for hiding losses, and still defend Blizzard despite losses returning in HotS.
On the second argument, it is sometimes claim that a global rank is too noisy, meaning there are so many players on the ladder that getting 12 points for a win would move you, say, 847 places up the ladder. Using a percentile and updating the ladder only once a week would eliminate noise.
But even if there is noise, so what? Game results provide useful information for accurately updating points and ranks. Should we avoid ever updating points because it adds “noise”? If noise is a problem, then what about all the noise in the current division ladders?
There are also claims that skill simply cannot be measured accurately with MMR and that there’s no way to account for uncertainty about MMR, so that global ladders are meaningless. But this is just completely wrong. The uncertainty about MMR, is already measured by the system, and I’ve suggested that it should be explicitly included into points by using MMR-1.96*sigma. And even if sigma isn't directly used to calculate points it still tends to reduce as games are played. Skill can be measured with good accuracy, as shown by the near 50-50 matchmaking SC2 achieves using MMR, and the empirical evidence from similar skill rating system such as TrueSkill which have remarkable success.
4.4 Clearly articulate league promotion and demotion criteria and remove the GM league
Currently, the specific criteria for league promotion are not known. Blizzard occasionally releases a table with approximate criteria. This one was only valid until December 21 2011. This table would be unknown to those who don’t visit the website, and it’s not even updated for the current season. Blizzard needs to be more transparent on league promotions.
Using the global ladder I’ve suggested, the promotion criteria for Masters, Diamond, Platinum, Gold, Silver and Bronze should simply be having a percentile greater than 98, 80, 60, 40, 20, and 0, respectively. Since the current requirement, MMR > a and sigma < b, is equivalent to MMR-1.96*sigma > a-1.96*b, where a is chosen so that 2%, 18%, 20%, 20%, 20%, and 20% of active players are in the corresponding leagues, it follows that the proposed promotion criteria is approximately the same as a current promotion criteria. It accounts for uncertainty about MMR. And it’s very clear. Similarly, sticky boundaries for demotions can be set, e.g. having a percentile below 97.6, 76, 56, 36, 16, 0 will prompt a demotion. Note that one benefit of excluding inactive players from the percentile is that it allows for the proposed promotion and demotion criteria to be fixed.
The GM league should be scrapped because it is fundamentally flawed and would become irrelevant with the introduction of a global ladder. Currently, it takes the top 200 players at the start of the season. These players stay in the GM league as long as their bonus pool is less than some threshold. Consequently, the GM league does not rank the current top 200 players, instead it mostly ranks the top 200 players at the start of the season who have remained active, even if their skill level falls out of the top 200. Even worse, if a player during the middle of the season becomes more skill than the players in the top 200, than he will be unable to enter the GM league unless someone currently in the GM league becomes insufficiently active. Blizzard’s attempt to prevent players from flip-flopping in and out of GM league has created this flaw. I have no problem if Blizzard wants to reward the top 200 players on the global ladder with a golden hexagon and a feat of strength, but they should be put in a Master’s league division.
5. Ladder Anxiety
By Blizzard’s own admission, the addition of unranked played should alleviate ladder anxiety, so there’s no reason to not introduce a global ladder. In this section, I outline some additional steps to deal with ladder anxiety.
5.1 Deemphasize ranks and emphasize points
Blizzard should remove the harmful and pointless obsession with division ranks, which together with the current bonus pool system, perpetuates the treadmill effect. The first pain-free step would be to display points more prominently and ranks less prominently since points tend to increase. As suggested in 3.3, ranks should be replaced in the matchmaking page and the score screen by points. The division rank and global ladder percentile should be displayed in the ladder page in the profile. Also, only updating the ladder once a week will significantly reduce the treadmill effect.
5.2 Produce a ranked games tutorial to explain how the design of the matchmaking system implies current season profiles cannot get screwed up so that there is no need to have ladder anxiety
The reason why I don’t get ladder anxiety is simply because I know how the matchmaking system works. Therefore, I understand that MMR is self-correcting so it’s not possible to stuff up your current season ladder stats. If I’m on a massive losing streak, I know that I will be matched to easier opponents, thereby returning me to my previous MMR. So this must be a temporary losing streak which is followed by a recovery, unless my skill has legitimately fallen. In that case, it is deserved. If I eventually improve, my points will be returned to the correct value. In the long run, the MMR is always right.
Moreover, overall win loss ratios are meaningless because the matchmaker creates games with equally skilled opponents, so it nearly always converges to 50%. The only exception is for very high (or very low) skilled players, because in these cases there may not be equally skilled players, which causes the matchmaker to expand search to find less (or more) skilled players. Hence, focusing on your overall win ratio or aiming for a greater than 50% overall win ratio is both pointless and in a sense, out of one’s control.
Blizzard should add a tutorial for ranked games where the adjutant outlines how the ladder and matchmaking system works. It should explain that the system has been designed in a way that implies current season profiles cannot get screwed up, as I’ve done here. Then there would be no rational reason to have ladder anxiety.
On a side note, Blizzard’s official explanations of the ladder system are an infantile joke, they're grossly oversimplified to the point of uselessness, lack basic details, scattered across the internet, and occasionally littered with outright untruths (a few of which have been pointed out in this post). This would be a good opportunity to come clean.
5.3 Allow an option for previous season history to be hidden
As explained in 5.2, it's not possible to screw up current ladder stats. However, the previous season history is unchangeable, and it's the only way players can permanently screw up their account. So allowing an option to hide previous season history should help alleviate ladder anxiety. While this change would be a loss of information, at least it's not current information (which would be unacceptable regardless of ladder anxiety), it’s old information.
5.4 Emphasize team games
Encourage new players and casual players to play team games before venturing into solo ranked games. This piece of advice could be included in the tutorial suggested in 5.2. It’s less stressful. We also know from official WC3 stats that most people prefer team games. They makes up for almost 75% of all games played.
6. Previous Suggestions
In my Battle.net feedback thread sections 5, 8, 9, 11 relate to the ladder system.