StarCitizen Server Meshing Architecture

by password-234 • 3 years ago

103 131 3 years ago

danpalmer3• 3 years ago

How does this relate to the technology that Improbable[^0] have built? I gather that their core tech is a massively distributed object graph designed to support the sort of interactions necessary for an MMO game to operate a global instance rather than shard worlds.

Improbable seem to have gone a bit... off the rails(?) in recent years, diving into Web3, metaverse, and defence contracting, but I assume the core tech is still there.

[^0]: https://www.improbable.io/

Negitivefrags• 3 years ago

Improbable is basically a joke in the games industry.

I’m not exactly an expert in them or anything, but the last time I looked at them at all (admittedly many years ago) the whole concept of what they do seemed both very expensive and mostly useless for real games.

Thaxll2• 3 years ago

It's not useless tech but they ran out of steam and most likely missed the train.

floren• 3 years ago

Improbable is basically a joke in the games industry.

More so than Star Citizen?

postexitus• 3 years ago

As someone who glimpsed inside for a little bit during the first days - it's a curse of having too many resources, or too few pressure points. They were well funded from the beginning (read their founding history) and got a lot of additional money throughout the way (well, including but not limited to SoftBank) - so they were free to explore their vision, and gosh did they take their time to do it. They had good ideas coupled with a great engineering team, but the problems they were trying to solve did not exist in the real world - or at the very least - the problems were not important enough that game companies wanted to throw money at it. 1M concurrent online players looks good on paper - but who really cares to play such a game? Or who wants to pay exuberant prices for it when you can have the same amount of fun through sharded servers. So, although their vision materialized to an extent (they had the means to do it), and they had the foresight to know they had to support the engine with enough tooling and partnerships and what not and they built that as well, they were not able to make money out of it. If they set out to set problems that really existed in the world (gaming or otherwise) they could have been profitable by now. Instead, they panicked, restructured and pivoted into the latest hype(s) - they still have the means to go on, but with that culture and "baggage" (they are not nimble like a startup anymore) I give them less than 10% chance to turn a profit ever.

danpalmer3• 3 years ago

Ah. Lack of Product-Market fit. Classic. Thanks!

doctorpangloss• 3 years ago

Unity ECS, which should really be called Unity “2”, sort of obsoleted Improbable’s approach. It is a base on which massive MMOs can be built correctly without the R&D cost of, well, inventing your own ECS architecture.

I don’t know if their Unreal offering is comparable to what they were doing with unreal.

tgtweak• 3 years ago

The fact this is not done by members of the dev team is pretty impressive both from the fact that they share this much information freely (on their youtube dev vlog?) and that someone was able to recreate the architecture semi-faithfully with that same info, including over-time.

neals• 3 years ago

I'm vaguely familiar with these topics. StarCitizen is a controversial game, if I'm not mistaken? What's this website about?

Cthulhu_3• 3 years ago

Back-end server technology, they want to make it a persistent MMO with thousands of concurrent players and effectively infinite objects in space.

But this MMO is the most expensive video game of all time (although there's others that have had more invested in them over time, I'm thinking of World of Warcraft or GTA 5, possibly upcoming titles like GTA 6), therefore the server technology also looks like some of the fanciest of all time. They have 1100 employees working on a game that's been in development for over 10 years now, funded to the tune of $600 million so far, with about 50-60 million still coming in every year just from pre-orders / donations / in-game purchases / ships for a possible game that may come out at some point.

TheRoque2• 3 years ago

The majority of the team is actually said to be working on the "solo" project which is Squadron 42. So they have an excuse for the sluggish development pace. Funny thing is, there hasn't been a single footage of SQ42 in 5 years, and you can't even buy the game anymore. There's a roadmap that says that the game is nearly done, but well, they seem to say this all the time.

infosecSnowman2• 3 years ago

Looks like someone mapped out the tech deliverables that SC has completed to meet their tech goals.

swarnie• 3 years ago

$600M buys you a lot of TBDs apparently.

If this game ever actually comes out and doesn't cure at least one type of cancer the weirdest online people are going to be furious.

ojkelly• 3 years ago

Sure not everyone who bought the game or a ship at this stage has enjoyed it, but a lot of us have. Even in its unfinished state there’s a lot of fun to be had.

And, part of the game at this stage is watching it develop.

The amount raised/sold so far is massive, but people keep buying because they’re getting something out of it.

lolinder3• 3 years ago

What features are built and playable at this point? By its nature, it's hard for me to disentangle the aspirational marketing from the actually-available content, and there's so much critical content about SC over the last decade that it's hard to know which criticisms still apply. What can you actually do in-game?

Mayzie• 3 years ago

I backed this game on Kickstarter back in 2012. I don’t care for all this focus on multiplayer, I was just in it for the singleplayer aspect. I wish they would just deliver that.

sumtechguy• 3 years ago

This game is weird. It is like not failing and failing at the same time.

sdflhasjd• 3 years ago

The game is the meta-experience of playing an early access game at this point.

yuppiepuppie• 3 years ago

I know the words on this site. But I dont know what they mean. "Quantum Economy Scaling", "Subsumption AI", "pCache". Can someone ELI5?

~Im assuming this has to do with a service mesh and kubernetes of some sort. But I'm a bit lost.~

Edit: Ah! Seems like this is a game. But I still have no idea what any of this means :)

Cthulhu_3• 3 years ago

Star Citizen is a big planned online MMO. This page is about their current and planned back-end server architecture, where they want to have thousands of players online in a single virtual world, along with all of the random items - from ships to bits of trash - that float around in space visible to everyone.

Ambitious and such; the only other game I know of that tried it was Eve Online, which has tens of thousands of players in the same world, but which is limited to hundreds of players that can actually see each other.

causi2• 3 years ago

You sure about that? There are several EVE battles that involved multiple thousands of players at once.

Goronmon• 3 years ago

This page is about their current and planned back-end server architecture, where they want to have thousands of players online in a single virtual world, along with all of the random items - from ships to bits of trash - that float around in space visible to everyone.

I would be surprised if they reached "hundreds" of players in a single instance. I don't think thousands is ever going to be attainable for them. Unless they are somehow able to make some sudden breakthrough that's alluded them in the last decade plus of development.

monlockandkey• 3 years ago

I have been in 1000 player battles in Eve Online. It is possible because the game operates in ticks (increments) of 1 second.

infosecSnowman2• 3 years ago

they're looking to use AI to simulate an economy, the "quantum" is just what they're calling their tech.

They have several youtube videos on their channel where their game director has come out to talk about these projects in a technical manner. It's pretty neat.

[I say these things as a backer patiently waiting for more game]

AlexandrB• 3 years ago

These are basically marketing terms meant to try to explain why after 10 years and $600 million dollars in "crowdfunding" the game is still in alpha.

gumballindie• 3 years ago

I too wish to find such gullible investors to fund me for 10 years of alpha building.

dfc• 3 years ago

Wow! I thought $600M was an exaggeration, it's not: https://robertsspaceindustries.com/funding-goals

droopyEyelids• 3 years ago

It's a game that will never be completed, almost a kickstarter as performance art.

The words they choose must serve the goal of their performance first, conveying technical meaning is a optional secondary requirement

m3kw9• 3 years ago

QES and SA, the latest in technology

meesles5• 3 years ago

There's a massive set of articles pertaining to the topics. You won't be able to grok all the terms from a simple graphic, it's a massive project.

m0llusk• 3 years ago

Star Citizen is a space simulation game in a large and persistent simulated universe.

Quantum is a system for simulating activity with economic impacts and determines how much of raw material or assembled products are available and where, how much things cost, how busy and how dangerous or safe space travel lanes are, and other such aspects of economic scaling.

Subsumption AI is used to drive NPC behavior up close. The game is supposed to have a population that is 90% AI NPCs that drive economic variation and action in general. This makes the game world seem alive even if there are no players currently with you in game. The NPCs have the ability to decide what to do, how to move around, make jokes and idle banter, and can use various equipment in the game as well as fight either in person or as spaceship pilots.

Not totally sure about pCache but it appears to be used on the server to capture changes to state in the universe that should be stored persistently. This is part of the server container streaming feature which allows servers to stream in areas and sets of objects as players approach and then stream them out when they are left empty for a time.

RNAlfons2• 3 years ago

Judging from the games performance, this looks probably better than what their dev team has.

Cthulhu_3• 3 years ago

The game was always built for "future computers", but then, that was ten years ago. They're really putting a lot of money and effort into reporting on their progress, after a few years of radio silence. But then, they have a lot of money to burn.

mattanimation• 3 years ago

indeed, haven't been able to finish a mission yet due to crash after bug after crash.

madsmith• 3 years ago

I’m running and finishing missions daily. There are bugs, yes, but it is imminently playable.

meesles5• 3 years ago

Have you built anything comparable? It's in alpha, making performance/presentation comments seems really out of place. If you were well-informed on the project, you would know that they intentionally put performance optimizations on the backburner, because optimizing tech they intended to replace is not a good use of time.

Besides, in the last 6 months performance has drastically improved, as anyone that actually plays will tell you. I play regularly, do a variety of delivery/bounty/mining/recovery missions, explore planets/stations, have funny interactions surrounding all of that with other players, etc. Rome wasn't built in a day.

RNAlfons2• 3 years ago

Do I have to build something comparable to be able to comment on the performance issues I have witnessed? I don't think so.

Do I have to accept all those excuses, which like those pictures of ships never run low in this company, to comment on the performance? I don't think so.

They promise a lot. They don't deliver. They break things which were already working and I'm still waiting. Those are clear signs of terrible Project Management and I think, I don't have to know all those cool insiders-excuses to be able to comment on this failure.

So please. Spare me your epigrams. They won't change anything about the facts.

empath-nirvana2• 3 years ago

It's in alpha, making performance/presentation comments seems really out of place.

This is a ridiculous statement. They've collected half a billion dollars from customers and have been in alpha for over 10 years.

chrisjj4• 3 years ago

Reminds me of the film director dismissing the view of a film critic who had never made a film in his entire life...

hotdogscout• 3 years ago

What is the state of the art for MMO servers?

jalino23• 3 years ago

got addicted to SC this year, spent an embarrasing amount on it. this Server Meshing they're planning is impressive if implemented

tiledjinn2• 3 years ago

"impressive if implemented" is a nice nutshell for star citizen, which on the whole isn't terribly impressive.

password-23410• 3 years ago

Its pretty impressive if you go in with an unbiased outlook

IF you have the hardware to run it and you're a scifi nerd its mindblowing the first time around, even after 3000 hours i still have moments when i'm mindblown. (Recently had a tank that shot my friends ship from the sky during an ingame event)

JJMalina3• 3 years ago

Over the past two years I've played maybe 10-20 hours of SC. I paid $45 for the starter ship and then made some money moving boxes and doing bounties to upgrade my ship and then have enough to buy a bigger one. The game is pretty fun if you're used to MMO style grinding. The only problem is that during the past two years there have been many server state wipes so you lose everything except for the starter ship you purchased. On top of that there have been intermittent stability issues which would cause you to disconnect in the middle of doing something. It seems that it's become much more stable. So yeah, you're at minimum paying $45 for a game that's alpha, which is questionable value. My justification for this was that there isn't really any other game on the market that gives you the same experience. Starfield is releasing soon, but that's all single player, and I don't think it has space flight sim combat like SC does. Despite this, it's still a fun game with a lot of things to do, however it will not be everyone's cup of tea. Check out https://www.youtube.com/@CaptainBerks and https://www.youtube.com/@Morphologis for a better idea.

I really hope server meshing pans out, but I'm skeptical that RSI has the chops to pull it off. The number of objects in SC is huge and the world is seamless. This not something that's entirely new though. EVE Online has meshed servers for persistence of tens of thousands of concurrent players in a single universe successfully for decades now.

mattmanser5• 3 years ago

There's Elite Dangerous and No Man's Sky, as well as Eve online depending on what you're looking for.

All have their faults, but there are obvious alternatives to StarCitizen out there that's not vaporware.

dabluecaboose• 3 years ago

There's Elite Dangerous and No Man's Sky, as well as Eve online depending on what you're looking for.

None of those offer all the things as SC even at it's current state. Elite Dangerous is closest and even then they're just different games inherently.

there are obvious alternatives to StarCitizen out there that's not vaporware.

You can download and play Star Citizen right now. It is, by definition, not vaporware.

dotnet004• 3 years ago

I think that of the two, Elite Dangerous is the only one that scratches a similar itch, especially in terms of the spaceship mechanics. But it too has suffered from severe mismanagement (pursuing features that got advertising hype over those that made the game fun, abandoning features and leaving them in a broken state, taking forever to fix bugs but being quick on exploits, poor communication and of course the general brokenness of the way they designed their multiplayer in the first place).

I suppose Elite's one saving grace is that it doesn't claim to be in alpha.

JJMalina3• 3 years ago

Whether the term "vaporware" applies to SC is kind of subjective IMO. The server meshing feature has been advertised for a long time and it still hasn't been shipped. Though they did add object persistence a few patches ago which really changed the game for salvaging.

I bought the game based on what I see people doing in it already, and it was fun enough for me. That definitely won't be the case for everyone and I recognize that. So to me it's not vaporware and there are lots of other people still enjoying the game today.

jcfrei• 3 years ago

Never played Star Citizen myself but follow the development closely. From my understanding it's the only one (of the ones you mentioned) that simulates actual planets, orbits and has seamless transitions from space down to the pavement of a city.

silentsanctuary• 3 years ago

Many (I would have guessed most) Star Citizen players are long time players of these other games! What SC offers just isn't replicated by anyone yet, but you can be sure that players of this genre keep an eye on the horizon for what other projects might be offering, on a quite regular basis.

wlesieutre• 3 years ago

The jump gate to Pyro was a very exciting prospect when they first demoed it four years ago...

https://www.youtube.com/watch?v=-tB3cark5lA

But I'm in a similar boat now, backed on Kickstarter almost eleven years ago now, and playing the game is just too much of a job.

In its current state, you spend 20 minutes riding elevators and trains around to get off a planet, and then crash into an invisible asteroid and explode, respawn and start over. And add a bunch of errands flying around the system to replace all your gear that blew up.

When it works it's very cool. But unless you have a lot of free time, not worth all the times when it doesn't work.

I hope they finish it, once it stops wiping progress and is possible to slowly accumulate longer term progress I'd be more willing to put in the effort. As-is, I'm not putting in any more money. They need some pressure to actually finish the damn thing (especially Squadron 42), and the bottomless fountain of crowdfunding with no publisher calling the shots clearly isn't working.

marknutter2• 3 years ago

Something that needs to be pointed out here is the fact that nearly all of CIG's resources are allocated to finishing their single player game, Squadron 42: Episode 1, and a veritable skeleton crew is working on the Persistent Universe (the MMO). Considering this, and the fact that most of the work done on Squadron should directly translate over to the PU, a lot of the skepticism over how fast development is going is misplaced. At least... I hope so.

bitvoid• 3 years ago

At least... I hope so

As do we all, but CIG doesn't make it easy to hold on to that hope when their transparency has all but disappeared. Squadron 42's store page has been removed from their website for months now, where any links to "pledge" lead to a 404 error. The only acknowledgement of it from them was buried as a reply to a days-old, user-posted forum thread where they claim it was removed for a price change. That was months ago. Meanwhile, multiple ships' prices have been updated.

lolinder3• 3 years ago

It looks to me like "server meshing" is a concept coined by SC for the architecture they have planned. The top page of Google for the term turns up only SC-related results.

EVE uses a different, simpler model where each system is a process, and players connect to proxy servers that keep track of which system they're on (and therefore which server they need to connect to) [0]. I'm sure there are layers of caching and other optimizations built on top of that basic structure, but that's just Gall's Law in action: a working complex system invariably evolved from a working simple system.

I don't think it's a coincidence that the architecture that I can describe in a single sentence is the one powering the legendary 20-year-old game, while no one seems to be quite able to explain what "server meshing" will look like except that it seems to have a lot of moving parts and still hasn't been finished.

[0] https://www.engadget.com/2008-09-28-eve-evolved-eve-onlines-...

hwillis2• 3 years ago

Star Citizen is trying to handle physics for objects in stations/planets/ships, debris, asteroids etc. A couple orders of magnitude more objects, I think.

Note that I don't play or follow either game, but I think you're making an unreasonable comparison.

rideontime3• 3 years ago

I've played 1,483.4 hours of Elite Dangerous in the past 2-3 years. You don't have to wait for SC to catch up.

Thaxll2• 3 years ago

And yet there is nothing really impressive imo in Star Citizen tech wise, the server is still slow, fps are low, not many people on screen and a lot of issues.

You should do tech improvment to enhance the gameplay, and currently I don't see many gameplay improvements that have real value.

Starfield maybe the first space opera fantasty that will deliver.

tiledjinn2• 3 years ago

Escape Velocity delivered on the fantasy just fine.

exo-pla-net• 3 years ago

There's an open source successor to Escape Velocity that's fun and continues to grow:

https://github.com/endless-sky/endless-sky

crooked-v• 3 years ago

I'm still waiting for the single-player Wing Commander-alike game that was supposed to be the whole reason to bother buying into this Star Citizen thing in the first place.

netule2• 3 years ago

It's the reason I insta-backed in October of 2012 but I haven't heard a status update in years.

password-23410• 3 years ago

There will be news about it next month during the annual citizencon. Whether its good news I cannot say though.

mrguyorama9• 3 years ago

Did you watch the kickstarter video? It was so clear that they did not have a good plan to achieve their goals, and very obvious Chris was obsessed with extremely minor technical details that didn't matter.

I loved the Wing Commander games, but they had people in charge who could actually direct a project. Chris Roberts can't get out of his own head. The video was a clear indication that he was going to ruin it.

VohuMana• 3 years ago

This is really cool, I play Star Citizen a bit every time they release a new version just to see what they are up to. Persistent Entity Streaming was tech they were working on for a long time, it had a bit of a rocky launch but seems to be pretty smooth now. Crazy to see how much went into shipping that.

For anyone curious what PES does, when a player places an object somewhere or for example their ship explodes, that object will remain on the server in the same place even if you disconnect. Seems fairly simple from a user perspective but required a lot of changes to make work apparently.

jmyeet2• 3 years ago

I'm honestly stunned how often I see the pattern repeat of telling the users they are wrong [1], particularly in this MMO space that I'd dearly love to see more games in.

World of Warcraft ("WoW") is the elephant in the room. It killed the previous elephant (Everquest aka EQ) but has reigned supreme in a shrinking space for 18 years. It partially shrank because it used to have a social component that has mostly been replaced by social media but there are other factors too.

But what I scratch my head at is all the bad competitors that came afterwards and predictably failed because they made product decisions. The most common one is focusing on PvP. MMO is an interesting genre because only something like 10% of the population at best cares about PvP. So right off the bat you've reduced your potential market by 90% without doing a thing. But no, you get assured that "our PvP is so good, casuals will want to PvP".

There's another factor here too: in a persistent world where your power increases, PvP players fundamentally want an unfair fight. They want to be rewarded for various grinds. Contrast this with deathmatch and battle royale games where everyone is basically equal other than player skill.

Anyway, Star Citizen has now raised (and spent) over half a billion dollars on the same bad premise of PvP.

Server meshing aims to create a single persistent universe but PvP, which most will want to avoid, is fundamentally unfair because the people with the best ships, paid for with real money, will win. This is such a fundamentally bad and limiting design, it's hard to comprehend.

Imagine instead SC had defined a small core game loop and gone for a Sea of Thieves like world, both PvP and non-PvP. SoT has like 8-12 players on a server and they can PvP. Interestingly, a lot of players don't want this so they monopolize a server and agree not to PvP. Even in WoW, which has faction PvP, the players have basically voted out world PvP by concentrating a single faction on each server with very few exceptions.

But SC is doing things like server meshing when after a decade they still don't have a core game loop. I occasionally check in on the SC roadmap and progress. Not because I'm interested in playing but because I like watching a car wreck in super slow motion. Constant missed deadlines, new (incomplete) features being promised (and a few added), more promised features to fix the bad previous features and so on. It's a disaster.

[1] https://www.youtube.com/watch?v=HMqZ2PPOLik

kcb• 3 years ago

Only the biggest MMOs, really just WoW and FF14, can deliver PvE content at a pace even close to satisfying players. It's expensive to develop and players can only play the same content PvE for a limited amount of time. It's just not possible for anyone but the largest dev teams.

I would like to see an MMO that really focused on a wide range of gameplay systems rather than content. It's been attempted before, one example I think is Archeage, but unfortunately ruined by monetization.

jmyeet2• 3 years ago

I have a counterexample for you: Classic WoW.

For those unfamiliar, WoW came out in late 2004 and has had some ~8 expansions since. Each expansions adds new lands, new cosmetics, new dungeons, new raids and new systems. Additionally, these expansions changed existing systems. So much so that what is now refered to as "retail WoW" is considered too complex with too big of a barrier to entry by many. It's partially why the player base has shrunk as the game has increasingly catered to the top 1% to 5% ta the expense of the casual player base.

There has been a rich history of so-called private servers. These are privately run WoW servers that stick to older versions of the game, particularly the vanilla game and each of the first two expansions. These are illegal and never last long.

At Blizzard's annual conference, famously this came up when an audience member asked about it and the head of the game said "You think you do but you don't" [1]. This one guy probably did more to move this cause forward than anyone as it galvanized the community behind this idea.

Then 4-5 years ago Blizzard announced they were releasing "classic WoW". This is basically (but not entirely) the original game with some changes. Some were understandable, some less so. But it was basically the original game.

This has been massively popular and continues to be, so much so that Blizzard just released "hardcore classic WoW", servers where your character is over when you die. Lots of games have hardcore modes but it's interesting in WoW that it may takes ~6 days played to get to max level and a lot of people still want hardcore mode.

But my point is that this 18 year old game with about the same graphics continues to be massively in demand. You don't need to constantly churn out new content to a large scale like you suggest.

Also, SC has over a half a billion dollars and continues to somehow raise a lot through ship sales (which, to me, is bizarre). They very much could afford to continually pump out new content. But there's still only one system with the second years delayed.

Studios like PvP because it falls into user-generated content. The players entertain themselves and that costs almost nothing. But they fail to acknowledge that they're, at best, catering to 5-10% of the potential market.

[1] https://www.youtube.com/watch?v=0Wrw3c2NjeE

usrusr• 3 years ago

Excellent observations, allow me to propose a semi-ironic counter position: at the time of its greatest success, SC had a very unique value proposition: instead of pay to win, it was pay to imagine winning. They were selling the daydream of getting one's entire attention and and ambition consumed in a game like a fourteen year old, to greying forty-years olds who grew up core gamers but who stopped a long time ago sinking quality hours into games. Now either hardly any hours at all, or just downwinding hours. The reality of PvP might be bleak, but an imagination of PvP is the best of all imagined gaming. Of course this form of virtual virtual entertainment can't go on forever because it's always been built on the promise of one day ditching one of the virtuals, but it's been quite a ride, one of its kind.

StarCitizen Server Meshing Architecture

Related Stories