League Meta Post

Meta Post time! Just for curiosity sake now that the initial beta league has finished and the players have base ratings I thought it would be interesting to push the top cut rounds individually into the system to see what happens at a fine grained level. The usual caveats apply - the formulas work best with lots of data which usually softens ratings changes. Single round results mean the system is guaranteed to increase deviations as the results will all be drastic whichever way it goes.

For this post I am going to look very specifically at the top 16 games because in seven out of the eight the lower seeded player won. This actually produced more  interesting results than the expected winner winning.

So for the majority of matches when both players started at 1731 (deviation 166) the drops were matched so the winner (for example) would gain 78 points and the loser would lose 78 points. Deviations would only increase by 9 points (due to the uncertainty generated by one match result). Ratings increase/losses were low because the system expects these players to be equal so a win/loss against the other will not impact placements that much.

For some games though the disparity between players was larger so now let's look closer at those games

Big Belly Jarelli (1847) vs Bill_jr (1565)

qkieu (1731) vs GravyK (1615)

Khift (1731) vs JoeFromCincinnati (1615)

Etaywah (1731) vs HidaMozi (1615)

The biggest change was the (at that point) undefeated BigBellyJarelli who lost a hard fought game to the lowest seeded bill_jrs Lions. BBJ started with a high rating of 1847 whereas Bill had the lowest of 1565. This means the most drastic adjustments occurred for these two players. BBJ dropped a whole 135 points to 1712 and bill increased 150! Points to 1715. As you would expect the deviation increased the most for these two players BBJ by 17 to 183 and bill by 13 to 190.  The difference between the loss/drop values is that bills deviation was quite high at start at 177 and BBJs was the same as all the other players at 166. Therefore the system was less confident of bills rating so pushed him higher when it gained more data. It was more confident of BBJ and though he lost a lot of points his drop was not so large as Bills increase.  Bill_jr was the eventual runner up so his low start point insured he rocked up the rankings to a 2nd place and 1787 rating finish.

The next interesting result was Qkieu's victory over GravyK. This was the one 'imbalanced' match where the higher seed won the game. Results wise we see the winner (Qkieus) rating increased by a much smaller 57 (compared to what appeared to be the standard mirror match gain/loss of 78). His opponent lost the same which, even though a smaller drop than most, pushed him much lower than the other low seeds due to the fact that all the other low seeds won. Though qkieu won his next opponent was another lower seed who beat him (the eventual winner of the competition ltmechanicus) so he then lost more points and actually ended up at 1712 lower than the rating he started at in the top cut, still enough for 6th place though.

The other two players who lost to lower seeded opponents above ( Etaywah and Khift ) were both 1731s who lost to 1615s. Since this was also 'unexpected' their drop was a larger 103 with their opponents gaining the same. In all 3 cases deviation increased by a larger 11. Etayhwah's opponent HidaMozi lost his next and his end position of 11th with a rating of 1654 is not much highter than his start point of 1615. Khifts opponent 'JoeFromCincinnati' managed to reach the semi-finals so both his next win was against a perceived stronger opponent and loss against a stronger opponent this was enough to place  5th with a rating of 1724  (5th place because the 4th placed player could not play in the final cut so a high start of 1731 retained a high position. This in itself is interesting as it showed the adjustment top 16 were only really enough to force the top 3 higher)).

This all shows the nuts and bolts of how the ratings work when operating with different start/end points and it is important to re-iterate that the ratings are not a measure of skill but a measure of demonstrated historical ability at winning matches. In gaming systems where the game is, usually, regarded as balanced with only skill being the pure differentiator (let us say Chess though even this has its factors i.e mental state, stress, health, location which can all impact performance) the Glicko methodology has proven itself better able to predict results compared to the more legacy Elo. L5R has far more influencing variables including deck design, current meta, a players predilection to play a single clan regardless as to clan strength, clan opposition (i.e if your deck has a 'difficult' opponent in Scorpion and all six games in a league are against Scorpion) , test decking in the league to hone a deck before a more 'valued' competition (the world championships are approaching fast) and more. This is just to say that the rating is more fluid than it would otherwise be.

Anyway thanks for reading..