HN user
HN hasn't been burdened with intelligent discussion for many years now.
One only needs a cursory scan of the top posts to recognize this; ~80% of discussion revolves around braindead slop peddlers.
IPv6 was obsolete by the mid-2000s, majorly due to the advent of roaming. It was designed on the rather fanciful assumption that its deployment would simply supersede IPv4, that every software/hardware vendor would cooperate, and we'd have a pure v6 network which would also replace the traditional L2/L3 layers.
Ofcourse legacy compatibility trumps all, along with the ubiquity of NATs and roaming and we're now just in the sunk-cost phase, being left saddled with a horribly bloated protocol (128-bit addresses was a marketing choice; not engineering) that solves no problems.
Those rates are peanuts considering that a decade ago saturating 40G, per core, was more than reasonable via standard userspace networking, with atleast a few copies in the datapath.
As someone who has implemented various transport protocols (both standard and custom) the biggest hurdle in layering atop IP will not be the gauntlet of WAN routers, surprisingly, rather consumer NAT devices.
One interesting case for a particular Netgear family of routers was that, while the traffic would survive end-to-end it was not without corruption - very specific corruption which took the form of zeroing the first 4 bytes. Given this aligns with where the src/dst ports would normally be, I suspect it was being treated as TCP/UDP albeit without the actual translation path taken.
This is really just archaic *nix world problems; the NT kernel has had actual asynchronous I/O since inception -- even 30 years later, io_uring is mere batching to the kernel upper-half (and is a carbon-copy, down to the particular terminology, of RIO)
Those numbers seem awfully low; even going via the relatively slow kernel/kernel stack, you can comfortably achieve around 2Mpps (64B) tx, per core (single flow), on ~2010 commodity hardware via standard Linux userspace APIs.
For reference, bypassing the kernel you can saturate a 10G link for ~14Mpps on a single downclocked 500MHz core with same class of hardware.
Stuck on phone at the moment; will check out the code later.
If it's any consolation: the vast, vast majority programmers should stick to single-threaded design, as is evident from all the atrocious multithreaded software out there of which 99% of the time would be faster without multiple threads (and surely far less complex)
Unless you know what you're doing, just pick the real low-hanging fruit, like throwing some threads at file block decompression.
Although not as elaborate as this, I have what one would call a "homelab" and it's for, well, testing and experimenting -- I write a lot of high-performance server/network-centric software (e.g, saturating a 100G link with 64B frames is a common test) and virtual machines just aren't suitable/capable most of the time.
Reverse engineering (which is not actually something I particularly enjoy doing frankly)
No, I'm quite the outsider to the community having only been involved in the game for a few years (besides as a teenager when it was released) and working alone.
The project in question --a platform for AoE2 multiplayer /w general game improvements-- is currently under development, although the core game/MP has been functional for over a year now and our little beta-testing group plays regularly.
The simulation is, rather obviously, what's being referred to with the spiral-of-death -- it's not unusual for instance in Age of Empires II to encounter transient periods where-in steps take an order of magnitude or more than the period they're simulating on less-capable machines (due to some O(n*n)'s in the AI system).
In the end this is still a decision regarding how to handle player's falling behind; too far and they're effectively taken out of the game as they're no longer interacting within a sufficient approximation of the current state.
Under what design is "high ping" going to cause the game to run at lower "speed"? The only possibility I can see is where-in the turn period is, for some bizarre reason, also the granularity of the simulation step and you're also adapting the turn based on player latency (in aid of fairness, usually)
Having implemented such a system myself, a few times now, it has in practice performed sufficiently well.
DE is garbage in general so it's not really a fair comparison.
Certainly true for local and/or controlled networks, however over the public internet where you're competing for bandwidth with many others over limited (and sometimes already congested-) links it's a rather questionable choice.
Good catch, Morten :)
I'm of the opinion that it's an acceptable concession that a brief pause occurs in gameplay to permit another player to recover from transient issues (network hiccups, brief machine resource starvation, or generally: stalling) in favor of fairness rather than punishing the player too harshly for factors that are frequently out of their control.
Consider for instance the situation where-in one player's simulation enters the spiral-of-death, under the policy that you're proposing this player is effectively removed from the game; I would say this is a substantially worse trade-off -- I know that I would personally prefer some mild annoyance of brief pausing as opposed to a player leaving the game (and in session-based games, as RTS's generally are, a team-mate leaving often ruins the entire game)
Abuse is indeed a problem that most games I've seen of this nature don't address, and while there's no perfect solution, it can be quite effectively curtailed. In the systems I've designed, the basic principle is that each participating player has an allocated amount of wait time which serves as a quota that is charged while-ever the game can progress/run, but cannot, due to them being in a wait-state (connecting/loading/reconnecting/stalling/explicit pausing/whatever). There's some extra logic for handling the oscillation of wait-state.
I've heard this repeated quite a bit, and while I haven't checked the AI scripts themselves (although I don't recall/imagine there being a script facility to give oneself resources..) I can say that, engine-wise, the only handicap behavior for AI that I've noticed is:
- On the highest difficulty ("Hardest"), AIs are bestowed 500 to all resources on next-age research completion. - Work rates for researches are skewed to be slower for the lowest-difficulty setting.
RTS-wise it's just been Blizzard games otherwise; Starcraft and Warcraft III, the latter of which I did the most work with, mainly multiplayer optimizations and writing a server implementation.
However, I have been entertaining the idea of supporting other deterministic (RTS-) games, adapting them the same way --rearchitecting the multiplayer system/code-- as I have done for Age of Empires II, providing a unified platform for these games. The first candidate that came to mind was the Red Alert series.
Similarly, a common technique used within notably the DotA community (of which's map didn't have such a tripwire) was to analyze the replay for what were termed "fog clicks", since for whatever reason object selection is part of the command stream and those using maphack would often, intentionally or inadvertently, select objects otherwise under fog.
Are these players interacting in any way? If they aren't then it could be more tolerable, but one player being more than several seconds behind others is going to lead to a rather undesirable experience, especially in the middle of a battle (and having most of their commands dropped due to being invalid equally so)
It is, in fact, inherent to this model. While there will be some tolerance threshold as to how far behind in the simulation a machine may fall, it still must be rather conservative (generally a few seconds), as elsewise the context (state-) sensitive inputs start frequently failing -- if I issue a command given at world-state time t, will it be meaningful/valid at state t2? The answer is where you get your threshold and it's always just a conservative approximation.
Not to any meaningful degree -- inputs (from humans) permutate the state. You can, however, compute say the initial world state, revealing the map/resources/ect. Ofcourse in this case you'd may aswell just look for some sort of maphack.
The problem is simply the lack of validation -- I detailed it a bit below in my top-level comment.
The only exploits that are inherently applicable in the case of a simultaneous deterministic simulation model are those of the information-revealing class (e.g, "maphack"). This is where a standard AH system comes in.
As someone who fixes up old games in my spare time, and Age of Empires II being one of them, I'll provide a bit of trivia about the game's internals:
- The AI system is not part of the deterministic simulation. This was surprising to me, and after contacting one of the original programmers it was explained that it was due to a desynchronization bug that the "AI and network programmers weren't able to fix it in time".
A consequence of this design regression is that, due to the AI now being authoritatively run by the designated host player, network congestion issues arose which lead to a clear series of progressively more aggressive optimizations to reduce egress traffic. This primarily consisted of a very simple filter (mentioned in the article) which dropped duplicate commands in the common submission path, meaning it applied to both local user and AI commands, along with batching of AI-submitted commands which would be flushed at rather arbitrary times.
I'll note that I've restored AI being deterministic in my project.
- A rather obscure determinism bug resulted from their compiler's implementation of a few CRT routines, namely fsin/fcos and a few others, which leveraged the specialized ISA instructions of the same name. The problem being that these transcendental functions are beyond the scope of the IEEE 754 spec. and ergo are hardware implementation-dependent. In practice, contemporary Intel/AMD chip families produce bitwise the same result, however those around the time of AoE are known to diverge on results to some small margin (as confirmed by an Intel engineer on a thread I came across while researching).
- The game employs a dirty-update system for rendering, not only that it's at a scanline granularity. This was something I was very pleased to see, as it's a exceedingly rare to see such an important optimization in games of this era (although common in earlier eras)
- There's some "interesting" naming conventions, one being prefixing member variables names with "value" -- there's even a "valueValue". Very little consistency in general in this regard, a reflection of independence between teams working on different components.
- While there are attempts at validation of input into the simulation (albeit woefully inadequate), it relies on ad hoc inclusion of PID (player ID) fields within commands. This is entirely useless, as this information is not authoritative and controlled by the players, permitting them to "spoof" the contextual information required for validation.
This is one of the more perplexing aspects of the engine, especially given the necessary information about the origin of a command is ofcourse available.
(As this information was later publicly published by other individuals I don't see a problem with elaborating on it here as I have)
- An example of missing the wood for the trees: session information goes through an ad hoc compression for its wire form (just bitpacks fields) to conserve bandwidth, however the architectural choice is to synchronize this session state by just having the host broadcast the state --dirty or not-- every 200ms, flooding the network pointlessly (atleast in the 18.8k days)
- While Age of Empires is somewhat notorious for its poor multiplayer performance (notably when contrasted with say, the recent "Definitive Edition"), and while its implementation of this synchronization model is certainly rather juvenile, it should be understood that the final MP gameplay issues are primarily due to the choice of a peer-to-peer topology over the public internet, which at the time was the most reasonable.
While superficially P2P may seem like it should achieve the lowest latencies for instance, the reality is that the primary determinant is the characteristics --jitter, delays and packet-loss, reordering, ect-- of the path between two hosts and a P2P architecture means there's n*n paths (network egress/ingress paths are typically asymmetric). In contrast, a server/client model you not only have far fewer routes, but datacenters are located at critical points in the network, roughly analogous to comparing travelling A->B along a freeway versus via the maze of residential streets.
- The state checksum "algorithms" involved are bordering on useless, atleast for state-tracing (such as when debugging desync. bugs.). They appear to have been devised by way of believing doing "a bunch of random bitwise ops" constitutes sufficient mixing -- a quick test demonstrated that csum collisions were not just possible, but occured sometimes for over 80% of inputs.
--
There's a lot more that could be said but I feel that's enough for now.