An Embarrassing Tale: Why my server could only handle 10 players

https://medium.com/@jasonchitla/an-embarrassing-tale-why-my-server-could-only-handle-10-players-3b83b6fa8136

by jachitla • 9 years ago

87 51 9 years ago

taormina2• 9 years ago

http://knckout.io/ is the link for anyone just wanting to play his game.

Also, is collision detection currently working? I mean, just trying to keep on the platform is actually pretty zen, but running into other players doesn't seem to actually do anything right now.

wongarsu2• 9 years ago

Collisions only do something if you boost into the player (press space).

julianj• 9 years ago

Or the up key-- It's a nice, simple game. Good job.

jsnell• 9 years ago

It depends on whether the players are boosting. I think it goes like this:

- If neither is boosting, the players just pass through each other

- If one player is boosting, they'll knock the other off

- If both are boosting, the players just bounce off each other

I find the controls extremely unsatisfying. Top down driving games (which this fundamentally is) really need a sense of momentum and drifting.

isaacg• 9 years ago

The lag is so high that I can't "boost" without going out of bounds and dying

jachitla4• 9 years ago

what region are you in?

camtarn• 9 years ago

Really enjoyed playing this game. It's really hard but pretty amusing. A good example of multiplayer gameplay distilled down to the very basics.

wongarsu2• 9 years ago

tl;dr: programmer makes game server for webgame, experiences massive CPU spikes, instead of profiling he randomly changes stuff until he finds the actual problem (not deleting past events).

jachitla4• 9 years ago

lol, nailed it

stormbeard• 9 years ago

It looks like he's still a student, so this is just the type of learning experience that breaks those bad habits. Hopefully his methodology will be a bit more refined after all this.

pagnol• 9 years ago

I noticed that currently json is used to encode messages between clients and the server and also a lot of information that is not actually needed in every update is included, such as the color of each player's car, their name or the number of lifes left.

How much could be gained by transferring only the information that is actually needed and using a different serialization format, perhaps protobuffers or the like?

synthmeat• 9 years ago

I got 2 to 4 orders of magnitude improvements across the board last weekend just by rolling my own binary protocol over ws, with world deltas. Didn't even bother to space-partition it to clients.

Using JSON as serialization format for real-time multiplayer game is being very silly, unless it's just a prototype.

jackmott• 9 years ago

yes json is crazy. protocol buffers are ok. but custom bit packed format is still helpful for twitch games

auspex• 9 years ago

Using JSON for a realtime game is extremely bad. Each request must not only be parsed (json-> object -> json) but each serialization requires a byte by byte encoding pass. Meaning that you're going to use a lot of CPU. Binary is a much better choice.

jlebrech• 9 years ago

could have used msgpack http://msgpack.org/

nullpilot• 9 years ago

Messagepack still contains field names as strings, which can easily double or triple your data, and explodes parsing time.

A game is a pretty good use case for predefined schemas like Protocol Buffers, which is used by CS:GO and Pokemon Go for example. Other games like slither.io use binary over websockets without any of these libs.

throwaway2016a• 9 years ago

I feel like a lot has been lost since the days when I was in my parents garage making TCP/IP games to run over dialup.

1. Binary was the only choice

2. You had to assume packets would get delayed (sometimes 10+ seconds) thus you had to do some fun stuff like try to use math to predict where the player would be based on what they were doing last time you got an update.

3. You had very little RAM to work with so growing memory would be immediately noticeable.

#2 was the primary reason in old game you used to see other players walking into a wall for 20 seconds before jumping half way across the map. Darn n00bs who couldn't afford 56k (joking).

I fondly remember the first time I played a multi-player online game I actually had to dial into my friend's modem over POTS. I then spent the next two months reading telephony documentation trying to figure out how to use the modem to do things like that myself.

</nostalgia>

reificator• 9 years ago

#2 hasn't gone away, the gaps have shrunken a lot, and we've gotten a little better at predicting actions, but it's still the exact same concept.

zacmps• 9 years ago

Definitely, but other methods, such as lockstep, are also used in some games.

bschwindHN2• 9 years ago

Don't worry, it hasn't been lost by game developers in the professional space. The article is presumably from a web developer implementing their first game. No offense meant by that statement, but if you're reaching for Socket.io (or even TCP/IP) for a real-time multiplayer game then you're probably going to have a bad time.

Thankfully we have free resources from professionals in the industry to keep us on the right track:

https://gafferongames.com/

monsieurbanana• 9 years ago

I remember reading it some time ago, and I gave it another pass, but it seems the only existing solution is WebRTC which, as the author himself admits, is extremely complex for the needs of a simple game.

And netcode.io, the solution developed by the author, which has a major drawback: it needs the installation of a browser extension to work (only chrome supported at the moment).

All things considered, WebSockets still seem to be way to go for most use-cases.

rayboy1995• 9 years ago

I use Socket.io just fine for my game, though I do implement it differently and send binary packets of my own protocol instead of JSON. I can see what you mean for new people in the field using unmodified events though.

slazaro• 9 years ago

#2 is called Dead Reckoning [1], in case anybody wants to know more about it.

[1] https://en.wikipedia.org/wiki/Dead_reckoning#Dead_reckoning_...

cuchoi• 9 years ago

Can you show the specific code that was leaking memory?

wolco• 9 years ago

Sounds like some form of load testing could help you pinpoint issues. Loader.io has a free tier.

maccard2• 9 years ago

Have you thought about using a profiler?

mijoharas• 9 years ago

Precisely this. The advantages that can be gained from detailed and effective use of a profiler can't be underestimated.

You no longer need to guess at where in your code something is taking up CPU or memory, just look. I'd say the ability to effectively use a profiler is one of the most important skills for a software dev (yet one that many people overlook).

maccard2• 9 years ago

I've found that in the length of time people have spent speculating about what's slow and what's not slow, I've measure it, and have proof. Another advantage of profiling is that it improves your intuition about what will be slow. When I profile my existing code add, I tend to see the same things come up over and over again - vector resizing, overly broad mutexes, wrong data structure choice. Seeing these every time means that I can often pre empt these issues, but I always profile it to get hard. Numbers before and after.

echlebek• 9 years ago

If you aren't using a profiler and benchmarks when analyzing performance problems you're just blindly stabbing in the dark.

hellbanner• 9 years ago

Check out https://developer.valvesoftware.com/wiki/Source_Multiplayer_...

age_bronze• 9 years ago

Just a comment on the gameplay, the characters are moving way too fast for the size of the screen, the whole game is just too unstable and random.

eridius• 9 years ago

What does OP mean when they say "an io game"? They reference a few games that have .io domains, but the domain has nothing at all to do with the technology or style of game (or even whether it is a game at all), so I'm a little confused.

tarr11• 9 years ago

Usually a free to play multiplayer HTML5 Canvas Game.

vortico• 9 years ago

There's not really a meaning of "io game" other than by example. It doesn't necessarily have to have a .io TLD. Here is a community with some examples. https://www.reddit.com/r/IoGames/

vertex-four• 9 years ago

Multiplayer long-running competitive games in which you kill/consume other players to become stronger, generally. Agar.io was among the first, and is mostly representative of the "genre" - people wrote a pile of clones, improvements, rethinks, etc, mostly with .io domains, so they wound up being called "io games".

stcredzero2• 9 years ago

People are welcome to see about how many players my alpha multiplayer server can support: www.emergencevector.com

The theoretical limit as configured is 460 concurrent players. Be sure to read the instructions, or you won't be able to shoot, and you'll drain your energy and become helpless.

probably_wrong• 9 years ago

Sounds like you found an usability issue. Have you considered changing the controls and/or warning the players when they do it wrong?

stcredzero2• 9 years ago

The former.

forkLding• 9 years ago

Quick suggestions on usability, maybe start off the game with players on a slower speed so they get used to moving around first and then gradually speed up by accomplishing actions that you want them to do in a training sort of manner so that players start learning and don't get confused.

Side note: I subscribe to Mark Brown's Game Maker's Toolkit on Game Design and Game progression, here is an example:

https://www.youtube.com/watch?v=2u6HTG8LuXQ&t=4s

I think it would definitely help

jachitla4• 9 years ago

I had thought of that. When onboarding a new user, they fly off the map A LOT. Only problem with this idea is, I think it would be a lot of friction for users that have played more than 1 or 2 games (assuming you have the same thing happen to all players on start).

I have been looking for a continuation mechanic though. Speeding up after some knockoffs sounds like a plausible candidate.

logicallee• 9 years ago

if OP is reading this, can you share your technical stack, frameworks etc that you're using. I feel like you kind of left this information out. (I don't even know what language you wrote this in.) Thanks.

jachitla4• 9 years ago

NodeJS. If you click on "Acknowledgements" on homepage of knckout.io, you can see some modules I used. You will also see what was used to kickstart the project. Email me at jasonchitla@hotmail.com if you have more questions.

sigi45• 9 years ago

Horrible game mechanics.

Took way to long to understand how to control it and collision doesn't work

shlant• 9 years ago

use the up arrow to "boost" and knock players off. There is no collision if neither player is boosting.

The controls could definitely use some work though

trollopTheJoe• 9 years ago

Poorly written article about a poor programmer writing a poorly written code.

Jyaif• 9 years ago

Space partitioning when there are 4 players is a bad idea.

An Embarrassing Tale: Why my server could only handle 10 players

Related Stories