SpawELO – small free matchmaking system for LAN parties

https://blog.spawek.com/SpawELO

by Spawek • 2 years ago

182 47 2 years ago

blog.spawek.com

digitalPhonix• 2 years ago

Has anyone tried something that isn’t based on ELO/trueskill for team-based games?

It feels to me like summing/averaging the ELO of a team to do matchmaking is:

a) just a hack to fit a square peg (team based matchmaking) into a round hole (individual matchmaking)

b) losing a lot of information about how intra-team dynamics would affect the result (as in player A and B play well together, better than their individual ELOs summed; but A and C play poorly together, worse than their individual ELOs summed)

pacbard• 2 years ago

You could assign ELO at the group level. It will increase the number of possible groups by C(N, g) where N is the number of players and g is the group size. It could work if the groups are stable enough.

If you have individual ELOs, you could pre-seed the ranking using averages and then let the algorithm take over.

KaoruAoiShiho• 2 years ago

Sorry for pedantry but Elo is a guy's name, not an acronym.

batch12• 2 years ago

Thanks, I didn't know what Elo meant and your comment helped me.

The Elo[a] rating system is a method for calculating the relative skill levels of players in zero-sum games such as chess or esports. It is named after its creator Arpad Elo, a Hungarian-American physics professor.

https://en.m.wikipedia.org/wiki/Elo_rating_system#:~:text=Th....

Scene_Cast22• 2 years ago

Agreed. ELO is a crude but simple statistical model, and its primary benefit is that it's easy to understand and reason about.

I'd love for there to be a multidimensional player skill vector (aka "embedding") that encapsulates more information as well as a more nonlinear model to go on top of it.

For example, in many games, a team usually needs a support player, but a single number isn't enough to capture that for the purposes of matchmaking.

Spawek• 2 years ago

Author here: Elo is commonly used and most gamers understand it. It's nice to be able to show each player their Elo.

I love the idea to use per-player embedding and to add a nonlinear model on top. I think it may improve the model dramatically, but I'm afraid that the model may be overfitting strongly as we have only 35 games in the historical for now. Anyway - I think I'll try this out.

dragontamer• 2 years ago

Yeah. Sports science is about evaluating individuals in a team and trying to come up with statistics that make team picking more reliable.

Moneyball is a good movie that dramatizes the dawning of this science.

Fantasy Football is a game version that's become popular for online gamblers.

-------

I've seen a paper or two floating around on the subject. But I've forgotten the names of them by now and I dunno any good references unfortunately.

Hope you find something in your search!

InvaderFizz• 2 years ago

Reminds me of a group of friends that played call of duty together in the CoD4/WaW days. Individually we were mid-tier players on average. As a group, we completely dominated. We all knew each other's strengths, weaknesses, and general tactics. There was often no need to communicate specifics because we knew where each other would be and what we were doing.

We were always giddy when we got matched with a clan, especially if they were a mouthy crew.

leni536• 2 years ago

I would guess that in team sports/esports betting some people use more advanced models, but they obviously keep it secret.

DigiEggz• 2 years ago

Excellent point about team dynamics. When trying to dynamically guess a group's Elo, it could be interesting to try and create a game-specific radar chart for each player and add those values instead.

tonyhart7• 2 years ago

its tried and tested by multiple online games used in dota,cs2,lol,val,cod etc

it is not perfect but close enough that it didn't matter tbh, I mean multiple company do it for good reason

Jach• 2 years ago

There are a bunch of GDC talks about matchmaking systems over the years too for those curious about things that have been done. For a successful game a lot of things have to be taken into account, not just skill.

bloomingkales• 2 years ago

I would say ELO doesn’t let you walk out of a team based game with anything other than a win or loss.

So, in sports, even if your team losses there will be players who will leave the season as All stars, or even MVPs (or vice versa, you are on a championship team but not critical at all to the team). Team esports truly is live or die by the win, and somewhat destroys the expression of players who may be league-wide top defenders/attackers/support. It’s not tracked or acknowledged.

It’s very brutal.

We need to track and factor in “advanced stats” like many sports do to some degree. You are not your ELO, but really your assists-per-game, rebounds-per-game, PPG, RBI, YPG, etc.

Match making would sort of fall into place more organically in that case possibly, since you can easily look at a team and see it needs more scoring, or more defense, versus … needs more winners or losers.

wbl• 2 years ago

CSGO at the pro level has ADR and k/d, but the stats for an igl are going to look a little different from the awper.

TheAceOfHearts• 2 years ago

Ultimately the only thing that matters is whether your team wins or loses. This should align every player to do their best in order to win at any cost.

In the last few weeks the League of Legends Worlds Finals took place. One of the teams who made it to Round of 4 was GenG. Their midlaner is widely considered to be among the best in the world, and their team has dominated their region. Despite that, he hasn't won a single worlds. The team that won has the most legendary league player because when he shows up to worlds, he locks in and wins by any means necessary. T1 was the fourth seed in their region, and they were one game away from not even qualifying to worlds. But once they made it to worlds they locked in and won the whole thing.

I think one of the key issues with matchmaking in team based games is that they encourage solo queue instead of forming groups of reliable regulars. Having a regular group of people to play with also reduces toxicity and it lets you have clearer expectations of your allies. The power of friendship is undefeated in team based games.

KK7NIL2• 2 years ago

We need to track and factor in “advanced stats” like many sports do to some degree. You are not your ELO, but really your assists-per-game, rebounds-per-game, PPG, RBI, YPG, etc.

WAR in baseball (https://en.m.wikipedia.org/wiki/Wins_Above_Replacement) has been a pretty successful attempt at this.

But it's besides the point, because the goal of a matchmaking ranking system (like Elo) is not to measure a player's skill but to pair player in such a way that there's a roughly even chance of either team winning.

Bombthecat2• 2 years ago

Heroes of newerth did something like this. They tracked gold earned, wards placed, kills, damage dealers etc etc etc.

At the beginning it was terrible. But after a year or two, it was pretty good fine tuned!

Buttons840• 2 years ago

I used PageRank once, seemed to work well. A win was treated as a link, points would flow from the loser to the winner. The strength of links would decay over time.

emmelaich• 2 years ago

This HN thread has some links and discussion that you might like.

https://news.ycombinator.com/item?id=26115665

https://mattmazzola.medium.com/implementing-the-elo-rating-s...

zeroq2• 2 years ago

I thought about it a lot and I'm afraid there's no single ultimate solution. It really depends on the sport and the variant you play.

Let me give you couple of examples: (a) foosball - you have a bunch of players who play both singles and doubles. What if you have two great defenders playing together in a team loosing to much weaker, but better suited for their positions, opponents? (b) counterstrike vs apex vs overwatch (or, to some extent dota) - what if you have a game where you can be easily carried by your team just by not sucking too much? In counterstrike one weak teammate (either lacking skill or a headset) can cost whole team a game, but in overwatch you can pick a supporting class and just chill at the back waiting for the rest of your team to win the game for you. (c) chemistry - you probably seen that at work as well, sometimes there's a synergy between two players that only occurs in that particular setup or it can be simply a matter of playstyle. (d) variants - in billiards you have different games that look similar but for some reason some players shine in one variant and can't get through in others.

chrischen• 2 years ago

There’s OpenSkill https://github.com/philihp/openskill.js

Also Microsoft’s TrueSkill

sdwr• 2 years ago

Everyone using ELO already does use something other than ELO. Game developers skew matchmaking based on:

- the amount of people in queue

- how long it's been since you last played

- total games played

- player reports

- how many cosmetics you bought (maybe)

The strength of ELO is how much it gives, for so little cost. One number representing everything. It doesn't perfectly explain the beautiful variety of nature, sure, but it's the most efficient possible abstraction. A single point value that captures 70% of what you need to know about relative skill.

Bombthecat2• 2 years ago

And how likely it is you quit because you lost or won to many matches.

Blizzard does it. Especially in overwatch.

It's called engagement matchmaking

kaichanvong2• 2 years ago

"something that isn’t based on…", Many of the CPL (Cyberathlete Professional League in eSports) qualifiers were based from online rankings for players. Normally round-robin (more common known swiss) format-of tourment systems applied; go from the previous ranks, it's often hidden for different reasons, however, you look at the most recent qualifier, who performed well, then try to order everything slightly differently offline-in tournaments, then report online.

The important thing to note, when the numbers look odd-number, removing players to join-in into the event, placing the limits!

Normal orders based on name, country of origin (sometimes these individuals travel far!), time of entering tournament (sometimes people late!), then getting it to play as well as possible sometimes based on how it looks… you want some kind of catch measure, to catch 2-players early on that would play normally in the final, having 1 going home early onwards. Is this a hack? Yes and No. Losing information happens when you don't apply planning, (1-day planning then more when possible?).

kaichanvong2• 2 years ago

someone wrote, it's Win or Lose, in these situations, sorry!

Event organisers, they like encourage the "good games" happening, when problems arise, it looks bad on every people there, the result OK, however this affects more. It is the event happening on time, people leaving at right times, everyone is OK, that people consider the "friend" aspect of tournaments, the professional points of view; this is "sportsmanship" online towards the real eSports victory.

Meanwhile the electronic groups have different "matchmaking" systems, I have even heard it is more matchmaker, it cannot be 100% perfect, however, people always try hard doing good.

joaquincabezas• 2 years ago

what about trying with Shapley values?

lucb1e2• 2 years ago

No idea what you're talking about so I looked it up

The Shapley value is a solution concept in cooperative game theory. It was named in honor of Lloyd Shapley, who introduced it in 1951 and won the Nobel Memorial Prize in Economic Sciences for it in 2012. To each cooperative game it assigns a unique distribution (among the players) of a total surplus generated by the coalition of all players. The Shapley value is characterized by a collection of desirable properties.

This is pretty much the whole article lede on Wikipedia but I have no idea what I just read. What's a total surplus in a cooperative game here, like how much wood your team has gathered in AoE? How does it help to "assign" that, wouldn't that rather be the result of a game? Can someone dumb this down?

palata2• 2 years ago

For tournaments, I like the Swiss system [1], which I believe is popular in chess.

[1] https://en.wikipedia.org/wiki/Swiss-system_tournament

jldugger• 2 years ago

It's also common in Magic and other card game tournaments, but slightly modified: "6 rounds of swiss, cut to top 8 the single elimination" is a common sight.

zeroq2• 2 years ago

Fun fact - our local community recently started playing billiards tournaments using a swiss system and it's really cool, but it's a tradeoff between fairness and the amount of players you can invite to a tournament. The swiss variants play at max 40 players with 6 rounds, but if you want to invite 100+ players then you have to run a 2KO ladder, otherwise the tournament would last a week. EDIT: The best part of swiss system is it's the best value for money, because you will play throughtout the whole tournament despite your results, and while you progress the opponenets will adjust to your level, so you can have fun no matter what.

GuB-42• 2 years ago

The problem is that a billiard is a big, expensive piece of equipment, and you are limited by your venue.

In chess, you only need a chessboard, which is cheap, and a table. In collectible card games like Magic: The Gathering, it is even simpler as people bring their own cards, so all you need is the table. For a LAN party, people usually bring their own computer and they have a spot for the entire event, so having everyone play at the same time is trivial.

lucb1e2• 2 years ago

From a quick read, after a first round where you play another team at random or so, in each round players are "sorted according to their cumulative scores and [play against] opponents with the same or similar score". At the same time, it avoids having to play the same opponent twice. I don't see how one decides how many rounds to play, but not sure if that detail matters

Reading the analysis section, this is how it compares to a knock-out tournament:

Assuming no drawn games, determining a clear winner [requires] the same number of rounds as that of a knockout tournament [...] Compared to a knockout tournament, a Swiss system has the advantage of not eliminating anyone [and] that the final ranking gives some indication of the relative strengths of all contestants, not just of the tournament winner. [However,] sometimes a player has such a great lead that by the last round they are assured of winning the tournament [such that it] does not always end with the exciting climax

palata2• 2 years ago

Chess tournaments I've participated in had 7 rounds. The swiss system is really cool because it is a kind of sorting algorithm, where players quickly end up confronting players their level. So if you're a beginner, you still get to play to get a better ranking (instead of "whatever, I won't win anyway").

haliliceylan• 2 years ago

Wow its very interesting, for me AI is still a big black box

toolslive• 2 years ago

Kaggle had (has?) a competition for rating systems.

https://www.kaggle.com/competitions/chess/discussion/107

There are plenty of rating systems out there that perform better than Elo.

Mkengine• 2 years ago

I only see the names of participants in the leaderboard, how can I access their methods?

shenberg• 2 years ago

Under the leaderboard tab, if the "Solution" column has an icon, it's clickable. 2nd place solution is by Jeremy Howard (of fast.ai fame), which I'd summarize as TrueSkill Through Time (Microsoft Research paper) + some overfitting on the public leaderboard (1st place was #26 in the public leaderboard).

SpawELO – small free matchmaking system for LAN parties

Related Stories