hpx7

Horizontal scaling is certainly a challenge. With traditional load balancers, you don't control which instance your clients get routed to, so you end up needing to use message brokers or stateful routing to ensure message broadcasts work correctly with multiple websocket server instances.

Hi HN, I'm the CTO at Hathora and wrote this piece. Happy to answer any questions

Hi I'm the author of this blog post and the CTO of Hathora. Happy to answer any questions about how we built our Stateful Router for WebSockets and other persistent connections!

WebSockets is built on top of TCP, whereas WebTransport is built on top of UDP. This makes WebTransport better suited for handling latency-sensitive applications like multiplayer games.

Does anyone know of any websites/applications that utilize WebTransport? The only thing I've come across are the Showcase demos from https://github.com/yomorun/presencejs

Hi HN, I wrote this article because I didn't find much on the internet comparing network performance between cloud providers. I was surprised to see the amount of separation between the top performers and the bottom ones.

We published our benchmarking script on Github[0], I'd love to see results from other geographies and perhaps even other providers like Linode and Hetzner!

[0] https://github.com/hathora/cloud-network-benchmark

If you're interested in using web technologies for your game, you may find the framework I've been working on to be interesting: https://docs.hathora.dev/#/

I've had success using it as the backend+networking stack in conjunction with game UI tools like Phaser and Pixi.js.

Hi there! I started a company this year focused on multiplayer server infrastructure. We also built a multiplayer game framework for Typescript that has gotten 400+ stars on Github in the past few months: https://github.com/hathora/hathora

Would love to connect and exchange notes about multiplayer development -- if you're interested, my email is on my profile.

Try https://hpx7.itch.io/ship-connect if you want to see a mobile friendly Hathora game (made for a different gamejam)

Edit: I realize now you're talking about desktop Safari. Star-jump doesn't seem to work on it for some reason, will have to investigate why...

I'm not the author of the game so not sure exactly what they meant by that, but that game uses Websockets which are built on top of TCP, so developers shouldn't have to deal with dropped packets (and our UDP implementation will have automatic retry built in as well)

For an example Hathora game that does client prediction and rollback, take a look at https://github.com/ourcade/flappy-bird-hathora

If anyone wants to try out a game I made using Hathora for a recent gamejam, you can check out star-jump: https://hpx7.itch.io/star-jump

It's a 2d multiplayer platformer using Phaser[0] for physics on the backend, and it's hosted on Hathora Cloud!

[0] https://phaser.io/

Hathora actually supports raw TCP connections in addition to Websockets, and we're very close to shipping raw UDP support as well!

For the web, we're deciding between using WebRTC or leapfrogging straight to WebTransport[0]. Either solution would be a thin layer on top of the raw UDP work

[0] https://web.dev/webtransport/

Hathora's auth story is basically bring your own JWT token. So you can implement your own custom auth as long as you end up with a JWT signed using your APP_SECRET in the end.

Websockets do actually support binary packets not just strings. But yes you lose control over the server side receiver part of the connection (we think most developers will be ok with this)

Hathora Cloud will be able to run anything that can be packaged into a Docker container and implement the Hathora Protocol (yet to be formally documented)

Hathora Cloud provides several important features specific to multiplayer gaming. To list a few:

1) Built in authentication/identity

2) Session/room based connections (with lobbies, matchmaking, etc)

3) Handlers for various transports (websocket, TCP, UDP)

Looks cool! Can I ask what tech stack you used?

It's close but TCP will retransmit frames rather than packing multiple updates in a single frame.

It's common for people to build this kind of retransmission logic on top of UDP (especially for networked games), it's sometimes referred to as "reliable UDP".

Exactly, you assign a sequence number to each update, have the client send acks to convey which packets it has received, and the server holds onto and sends each unacked update in the packet to clients (this is an improvement over blindly sending N updates each time, you don't want to send updates that you know the client has already received).

If the client misses too many frames the server can send it a snapshot (that way the server can hold a bounded number of old updates in memory).

I've been working on my own realtime networking engine[0] and I think there are a few important points related to network syncing that are not mentioned in this article:

1) Bandwidth. The users internet can only handle so much network throughput, so for fast paced games (where you're sending data to each client at a rate of 20+ frames per second) it becomes important to optimize your per-frame packet size. This means using techniques like binary encoding and delta compression (only send diffs).

2) Server infrastructure. For client-server games, latency is going to be a function of server placement. If you only have a single server that is deployed in us-east and a bunch of users want to play with each other in Australia, their experience is going to suffer massively. Ideally you want a global network of servers and try to route users to their closest server.

3) TCP vs UDP. Packet loss is a very real problem, and you don't want clients to be stuck waiting for old packets to be resent to them when they already have the latest data. UDP makes a major difference in gameplay when dealing with lossy networks.

[0] https://github.com/hathora/hathora

Looks very interesting. Is there some example Adama application source I can browse to see what typical code looks like?

1) The client and server are currently located in the same repo but I'm planning to make it so that you can run them separately. This will let you do things like write a new frontend for an existing deployed Hathora backend which you don't even own (as long as you have the hathora.yml definition for it).

2) There is a managed cloud coordinator (load balancer) which Hathora apps connect to by default, so you don't need to do anything for that. For the distributed file system I've been using https://aws.amazon.com/efs/, and each cloud provider as their own version.

3) Aren't the photon products tied to Unity?

Would love to chat in more detail in Github or in the discord: https://discord.gg/6nVdeCBffR

Definitely, once I add support for C#/C++/GDScript as language targets that is the next step

Someone on the discord server already hacked together a multiplayer Godot game using Hathora!

I'm not sure I understand the "Hathora-in-the-middle" piece -- Hathora isn't quite a Backend-as-a-Service like Firebase if that's what you were thinking. Hop on the discord server if you're interested and I would be happy to chat in more detail.

The way I've thought about preventing cheating with Hathora is to make it server authoritative, and make it as simple as possible to validate client actions in the server.

Yeah I'm curious to see where people bump up against the limits of the yml types DSL. I plan to make a UI on top of the yml to make it a bit easier to work with as well. I think the popularity of Protobuf as a format shows that people are willing to learn a new API format if it adds enough value.

Thank you for the feedback!

Thanks Justin!

I am familiar with Agones and I had briefly looked into leveraging Open-Match as a matchmaking option for Hathora.

To me, part of the value Hathora adds is from being an integrated and easy to use solution where you don't need to figure out how to put the parts together in order to develop your game.

I think where Hathora shines today is in its ease of use, both in getting started and in managing complexity as you add features.

That being said, there are a number of features that make Hathora performant and easy to scale for realtime multiplayer games -- you can read about some of them on the architecture page: https://docs.hathora.dev/#/architecture

I plan on adding benchmarks in the coming weeks to better quantify perf and scalability

I've had decent results with just using client interpolation so far, you can try out pong, ship-battle, or among-us to get a feel for the latency.

The latency will be even further improved once I add UDP support, and I also plan to write a library to make client prediction easier with Hathora.

Not sure if you saw it, but I wrote a little pong demo with Hathora in under 200 lines of code: https://github.com/hathora/hathora/tree/develop/examples/pon...

Generating a Lua client should be very reasonable. If you'd be interested in contributing (or just want to follow along), feel free to jump on the discord!