oscargrouch

Together with the author's distaste for Rust

As someone who understand the sentiment, I wouldn't call it distaste for Rust. It feels more as Rust not being the right tool for the job and to be honest I have the same feeling here. Rust is great for a bunch of problem domains, but it doesn't feel to be the best tool for this specific problem domain. It's always about the choices a programming language makes, it make it good for some use-cases and bad for others and there's nothing wrong with it. Also as a cautionary tale about panacea programming languages, Java once tried it with great success to be "the one" language to rule them all and we can see that even though it was a popular choice in many domains, slowly other fitting solutions to specific problem domains took over. Proving that there is no such a thing, unfortunately for us, but some languages gets closer than others (Rust being one of them).

Well if anything take as a compliment. As a C, C++ (and some Rust) who lately is enjoying Zig, I think Zig is the only programming language positioned to convince system programming die hard C programmers to use another programming language with simplicity and power backed in.

But completely agree. Its a perfect replacement for C++ and I would say the natural spiritual successor of C.

I gave up using Rust for new projects after seeing the limitations for the kind of software I like to write and have been using Zig instead as it gives me the freedom I need without all the over-complication that languages like C++ and Rust bring to the table.

I think people should first experiment see for themselves and only then comment as I see a lot of misinformation and beliefs more based on marketing than reality.

Thank you very much for your wonderful work!

We are in the "system engineering territory" and as such it might have more to do with the way the runtime is designed and how the javascript native runtime does things than the compiler optimizations. You have to measure syscalls, memory access, cpu cache locality and a bunch of design decisions that in the end contribute a lot to the running time. So depending on the decisions taken, it can easily happen.

My take on this as someone that professionally coded in C, C++, Go, Rust, Python (and former darlings of the past) is that Zig gives you the sort of control that C does with enough niceties as to not break into other idioms like C++ and Rust does in terms of complexity. Rust "breaks" on some low level stuff when you need to deal with unsafe (another idiom) or when you need to rely on proc-macros to have a component system like Bevy does. Nothing wrong with this, is just that is hard to cover all the ground. The same happens with C++, having to grow to adapt to cover a lot of ground it ended up with lots of features and also with some complexity burden.

In my experience with Zig, you have the feeling of thinking more about systems engineering using the language to help you implement that without resorting to all sort of language idioms and complexity. It feels more intuitive in way giving it tries to stay simple and get out of your way. Its a more "unsurprising" programming language in terms of what you end up getting after you code into it, in terms of understanding exactly how the code will run.

In terms of ecosystem, lets say you have Java lunch, C lunch and C++ lunch (established languages) in their domains. Go is eating some Java(C#, etc..) lunch and in smaller domains some C++ lunch. Rust is in the same heavy weight category as Go, but it can eat more C++ lunch than Go ever could.

Now Zig will be able to compete in ways that it can really be an alternative to C core values, which other programming languages failed to achieve. So it will be aimed at things C and C++ are doing now and where Go and Rust wont be good candidates.

If you used Rust long enough you can see that while it can cover almost all ground its not a good fit for lower level stuff or at least not without some compromises either in performance or complexity (affecting productivity). So its more in the same family as C++ in terms of what you pay for (again nothing wrong with that, is just that some complex codebases will need a good amount of man-hours effort in the same line as C++ does).

Don't get me wrong, Rust can be good at low level stuff too, is just that some of its choices make you as a developer pay a price for those niceties when you need to get your hands dirty in specific domains.

With Zig you fell more focused on the machine with less abstractions as in C but with enough goodies that can make even the most die-hard C developer think about using it (something C++ and Rust never managed to do it).

So i think Zig will have its place in the sun as Rust does. But I see Rust taking more the place where Java used to be (together with Go) + some things that were made in C++ where Zig will be more focused on system and low level stuff.

Modern C++ will still be around, but Rust and Zig will used more and more where languages like C and C++ used to be the only real contenders, which is quite good in my POV.

What will happen is that Rust and Zig programmers might overlap and offer tools in the same area (see Bun and Deno for instance) but the tools will excel on their own way and with time it will be more clear into which domain Rust and Zig are better at.

I worked on something in this space[1], using a heavily modified Chrome browser years ago, but I consider I was too early and I bet something in this lines (probably simpler) will take off when the time is right.

Unfortunately I got a little of a burnout for working some years on it, but I confess I have a more optimized and more to the point version of this. Also having to work on Chrome for this with all its complexity is a bit too much.

So even though is a lot of work, nowadays I think is better to start from scratch and implement the features slowly.

1 - https://github.com/mumba-org/mumba

I like your comment because it really make the root point here more evident..

Different strategies with different outcomes, it's hard to compare anything against the US which is an outlier, but there are some connections between tech and the cultural industry (especially movies).

And also as your point make more evident, of course that making the movie industry more sustainable and able to attract attention somehow is wanted. But giving there are people that like to eat more sushi than pizza, and giving the movie industry can shape itself while shaping its customers at the same time, i wonder if there's not a good way to mixing both (i mean hollywood could have been more bold in that regards instead of focusing too much in shaping its "milking cow")..

Anyway art is against "useful" or at least should have such a freedom that it doesn't even need to abide to useful.. Movies are a wonderful medium exactly because it's on this thin line between be art and something "useful" somehow.. making it a very sophisticated art form. (James Cameron comes to mind in mixing well both worlds, but guys like Denis Villeneuve are the ones that mix it really well in my point of view, bringing more sophistication to the table of popularity)

Somehow technology in general falls into those same traps. It must be sustainable, but to keep moving forward it also needs not to kill creativity. That's why i've mixed "maximizing the shareholder profit" into the equation to show how despite the fact that capital was paramount for the success of the endeavor in the first place, it can also be the source of its ruin when it makes the endeavors to lost what made them successful in the first place because they are focusing too much in just one vector (it actually kind of works great for industrial endeavors, but its not hard to see that more sophisticated industries like software and cinema, it doesn't live to it's full potential).

One of the reasons films are popular is by reinforcing a pavlovian behavior, where the movie industry learn what people like to see and give them that in return in a formulaic manner.

It works like a self-fulfilling prophecy, so once the formula of the popular of a given period is found, movies are made repeating those patterns to attract paying costumers (movies are one of the only industries who can mold their customers to become their customers).

Particularly i don't think popular convey into best, sometimes i'ts not even good, its just a Pavlov dog expecting to be entertained. So answering your question if people watch movies to be entertained, yes that industry works, as the profits show. But i disagree this industry create good movies as in the form movies were meant to be.

I don't mind formulaic movies, i don't judge people that want to be entertained as i was one of them and i still am from time to time, but once you dig down in the the cinema rabbit hole your sense of good changes quite a lot.. its like once you learn to taste more fancy food, with a myriad of tastes and colors a macdonald's does not taste the same. Or getting back to program in C after learn to program in D :)

Best as in movies for entertainment purposes, yes. Best as in sophisticated movies, i dont think it's the best system. I don't know what would that be.. and i bet it probably would need to be a "capitalism 2.0" that mixed with other things, but the system is kind of broken and until we really fix this we will need to live with economical crisis from 10 to 10 years..

Watching pre-code hollywood you can see how the movies from the US from that era were not far away from the european movies in terms of brightness.

Once i knew about this i understood why most of the hollywood movies turned out into the brain damages in comparison to the european cinema.. this together with plot based movies + capitalistic objectives being considered above artistic values and originality, turned things worse.

The only reason some US movies have artistic values is or because the film-makers are defiant of the system, rule-breakers, managing to bend the system into their favor or the money-making machine captivated some talented foreign film-makers.. But of course giving the system keep draining and eating itself, its never enough, and there are no roots nor a organic and sustainable growth.

But the system works in a sort of vampiresque form, so once there are no more talent to suck, or the talents are not "selling out" anymore, all the system goes down with time..

I love to watch the cultural and cinema areas because there are a lot of lessons to learn from the perspective of technology, because our fields are not that much different once you see the patterns..

I always tough that the US capitalism need some sort of balance to not always become imbalanced because its giving too much focus on profits.

Technology companies that were once inventive turn into to "give more profits to the shareholder" machines and eventually lose what actually gave them the edge anyway. Apple for instance didn't learn its lesson and its repeating the same errors of the past unfortunately..

I know it was about "pre-code" and "what it have to do with it?" right? But i focus on this to learn about how technology can blossom and other controversial things like "what the nazis did right" (*) in terms of technology to understand more about this..

(*) - My current answer to this is that the nazi were there just in time to collect all the science that have been promoted since the XIX century. So basically the nazis were there to collect and use all that scientific prowess to forge their war machine (of course they were also masters at giving people meaning and big goals/dream big).

FreeBSD have a lower memory footprint and their network (now together with their filesystem) make it a better server than Linux if you want a better use of your network bandwidth.

I love FreeBSD and would prefer to use it as my desktop instead of Linux, but for servers people should also consider FreeBSD as a great option for a better use of computational resources. (And all this even when Linux have zillions of smart people working hours and hours to optimized it which FreeBSD cannot afford to)

Now that you are saying this.. take a look everywhere, it's all like this.. every country turned into the worst versions of themselves, but even the worst of UK its kind of not bad in my opinion, compared to other countries.

At least from an outside view, it's the UK now that is showing a good leadership in the current worldwide crises. While the US, France and Germany are completely lost, especially on how to deal with a "strong-man".

(Meanwhile nobody remembers Cameron or that guy that was a pure puppet of the US in the forced Iraq war, and they were the posh gentleman everyone was expecting)

it just took me to scroll down on this same thread to find one sample

* 11 hours ago | parent | prev | next [–] Rust will quickly replace C in the kernel, I have no doubt about it

I've have seen tons of such a comment in all sort of products when the matter is discussed around here and elsewhere (twitter, reddit, you name it)

If you want to really get serious about this, i can feed this comment sections with tons of evidence over the course of the years.

You are fighting an imaginary opponent.

Yes, i'm sure there's an imaginary opponent downvoting my comment.

Also there's a lot into my comment, and people are not even noticing it in the whole context.

If there's no reason, why people get so upset? just move on if you are not being mentioned, as it will clearly be the case if i'm talking about "imaginary opponents"..

The one proposing it is a long-time kernel contributor.

I'm not saying anything about people working on it specifically, if you read my comment, there's a clear separation between serious people and the hype crowd (which is not just RIIR, but now also cryptocurrency fellows, etc).. i can't say where the people working on this fits, i don't know them. Don't know from which part of my comment you took that conclusion.

There are quiet, clever, serious people doing the work, like Hoare, Matsakis and the people that are real enginners, i have all the respect for them (and im pretty sure Rust have tons of such a people). To be fair, all languages have all kinds of people but i don't know what happen to some of them that tend to attract a certain type of people more than others, like the feeling a got from Haskell community more often than others (but given the community was much smaller)..

For instance talking about culture, C succeeded exactly because it was a pragmatic language very simple and efficient like their founders to get things done. With this culture, things happened to be done around the language and we have the ecosystem we have today.. it's a great hacker spirit of more humble, hard-working, behind-the-cameras sort of people which i sincerely miss in the days of instagram, tick-tock and tech celebritism.

Your example is quite funny and also represents how i would approach the matter of becoming a linux kernel developer.

Even if C is not my primary language of choice, i would definitely try adapt myself to the ecosystem and not the other way around.

You have all the knowledge of other peers, manual, books, all the libraries, the whole ecosystem.. this cant be replaced.

Also there's something else about C nowadays, is the lingua-franca, the latim (or english) of programming languages. We use it to expose api's to others in any other language that want to consume it as a library.

There's something about culture that people often forget in tech.. it's the real backbone of any project that it's on its own feet.. and when you want to enter in a community you will be better of if you learn and adapt yourself into this community culture instead of creating cultural clashes into the community and try to overtake it (be it hostile or not).

People should be aware that this effort will make it possible to create rust-based kernel drivers and that's it. the RIIR folks are delusional and hype fueled and its better if the sane Rust community get away from them or start to get them back into reality as i bet they are not willing to expend 10 or 15 years of their lives rewriting big and complex piece of software for a likely no return as people will tend to keep using the software the have more community and that are stronger.

It's a much better approach for Rust or any other programming language to become research darlings and eventually become the primary ecosystem of a research OS that went well and is the thing that will replace Linux. The language alone wont do it, it must be able to be a contender to UNIX and POSIX, and whatever language that is in such a system will probably be the one that will become the dominant one in such a ecosystem.

Also another good approach is to virtualize the Linux Api like gvisor does is userspace or the fuchsia OS(and even FreeBSD) does in the kernelspace. So that you can create your OS and kernel in the best way you can looking ahead, and have this Linux compat layer where applications dont even need to be aware they are not actually running in Linux.

i was born in a city near a german colony in the south of Brazil where a lot of germans from the 20's, 30's and later 40's arrived (which is unusual giving german colonies in Brazil are mostly from people of the late 1800's).

Anyway my father swore that he saw "an old man that looked a lot like Hitler without the mustache" in the 80's in that colony.

And its funny because its like those stories about people that have seen alien ships, you know is very unlikely but you never stop to wonder, "what if..". Well now there's no doubt that it was just a Hitler lookalike (poor man).

Also, one thing that always looked of in those theories is the fact that a man with the psychology of Hitler would be able to live a very simple life with such big regrets and big dreams smashed, seeing how much he was hated (remember, he was a narcissist) til old age. It always seemed unlikely to me, "not like hitler", while the suicide seems much more believable.

I think Gemini will have people following it and developing for it, but i think it could have been a little more ambitious and also took tech history a little bit more serious as to not take too much out of what is actually a improvement done on the web that could be a soft fork of the current web with a lot less things, but not as simplistic as Gemini is.

So i don't think Gemini is "it" yet, but at least i liked the audacity and the way of thinking, its on the right direction and actually it should be people like Tim-berners-lee that should actually be leading this, but unfortunately they are lost..

I guess its all coming down to mnemonics, aiding our memories and communicating.

Sure this is the "state of the art", but despite the fact that pure language notations might be even worse, i cant help to think that people thinking like the parent might find something even better.

Maybe something inspired by braille notation or something that is invented while trying to understand how our brain works (just speculating here) will be even more expressive.

I actually like seeing an adult be bothered by the fact that the same symbols that turn science more expressive are also the reason that there's a big ladder for newcomers to understand whats being expressed given its all very arbitrary (someone in the XVI century choose a random greek letter to represent X).

Imagine how much science would improve with more "brain power" being also able to try to solve some problems given there are less arbitrarity..

The point is that a lot of sites out there don't need anything more than the web tech of 2 decades ago. With more people being aware of that and using browsers like this, the Google monopoly may weaken.

I always thought this was a way to get out of this, that agrees with your vision. First you'll need a powerful force to take back control on defining web standards.

This new web standard would be a very simplistic one, making possible for two or three (or even one) people to create new browsers on top of it.

But this movement would need to be so strong, that it would be possible to make a dent and start to lead a new way. And this is the one of the most hardest part. People on this movement would need to be prepared to fight for at least 10 years without loosing its faith until the killer app of this movement should be able to be at a market share bigger than Safari and taking over the Firefox position.

This movement could use the hype akin to Rust folks to navigate the harder first years. Its a possibility but it's hard to become possible. Another thing to notice is that the Firefox position is getting weaker, but the most probable candidate to take over is the chrome-based Brave.

So unless a "black-swan" event happen it's hard to see a big change into this in the coming years. BTW it's even more likely that the browser standards get stronger by completely taking over the mobile applications..

The problem is: who's gonna use it?

Sure, it's good to have more options but, the deploy base of browsers like Chrome are massive and IE6 only lost its position by being careless and stop evolving (and lets not forget it took years), which is something that is not happening with Chrome.

Just to make clear, this is not specifically directed to you or your project, which is a great thing to do independently of the outcome. I'm saying this more as in a general perspective as we are pretty much sitted on the state-of-the-art of browser engines and its hard to see something else taking over except for a "game reset" scenario (outside of the web).

I've have a answer to this same thread specifying something i'm working on, so in case you are curious, you can have more info by reading the details i gave in the other answer, and the way to solve this problem is through indirection.

You don't expose your data layer directly to the consumer, you expose an api that will resolve to one, two or several databases from one or many peers. The indirection allow you to define your rules and use your data layer in a way that fits your application goals in the best way possible.

So whats is immutable is your application, api and initial data, which you can mutate at later stages through other torrents or by consuming other api's from other peers and mutating your initial database state.

The problem is, the current browser is not meant for this, Javascript is not meant for bigger and complex applications (of course you can do it, but..)

I've built something akin to this, but with the idea of applications distributing API`s where the databases are also torrented but working behind the api's, so that developers can build basically anything.

In my case i've implemented a new "browser" based on Chrome that allows this to work, without having to resort to browser-only infrastructure (for instance applications can dodge Javascript and also call RPC api's from other applications directly).

The applications and the applications data are distributed over torrent and managed to work together in the same environment as a flock where one app can consume its own apis and also the api's from others.

    service Search {
      rpc doQuery(string query) => (array<string> result) // the access to the sqlite db from torrents is encapsulated here
    }

---

With COVID and the rise of remote working there are a whole lot of companies where the developers are the asset, the commodity, where those companies are just the middle-man receiving for the work the developers do and getting the big part of the cut in the process.

Those are mostly the "sweat-shops" that hire developers that can work for less for several reasons. I see that in this particular case this solution might be very welcome to assess new hires.

But for more high-profile jobs and companies, the company that eventually follow this path will probably end with low quality hires, unless of course it just create crud's anyway.

Because this is a self-fulfilling prophecy in the end. Once the candidates know what they will face, they will train until they get good in that game, which most of the time doesn't mirror the qualities required for the job.

Is RT Linux a good (or at least a good enough) option for realtime OS's compared to others who where designed as RT kernels from the start?

It's there. For instance if you want to retain a reference to a 'WebFrame' for instance, you can create a wrapper object which is the one you directly control the lifetime and have a 'Persistent<WebFrame>' as a property of your wrapper to hold the blink reference.

As long the smart pointer is not destroyed (with the destruction of your wrapper or holder) the web frame object is guaranteed to retain a reference, making the object alive.

Otherwise you can retain Weak<> and Member<> smart pointers for things that your object dont own, and of course, in the case of Weak<> you can expect it to be collected in the next scheduled GC job (or anytime). And in the case of Member<> is not a strong reference to the object as in Persistent<> so you dont own the object's lifetime, but you retain a reference so the object should not go away while you hold it.

To see how serious is the commitment to this scheme, you can just explore the Blink codebase to see that this scheme is actually used internally as a way to control lifetime between objects, and not just as a API thing to be used from consumer projects (unlike V8 which vends a different API for consumers of the VM from the one it uses internally).

Foe anyone interested, there was a port of Webkit/Blink where the web objects retained by javascript are garbage collected outside of V8, using blink GC to destroy those objects.

The way it works its through smart pointers, where for instance you say how the reference to that object is retained according to the object that it references.

The good side of this is that other programming languages besides Javascript can deal with blink objects the same way Javascript does (This feature was called 'oilpan').

It's this feature that makes possible for my project to have web-based applications in Swift, for instance, and would allow to plugin the Lua VM or Jit in the same way and still be able to be a first-class citizen of the webkit API as Javascript is.

I feel like this would enable a more sensible choice:

I have a working solution for something like this that is based on Chrome

https://github.com/mumba-org/mumba

The applications developers publish native, Swift applications, where Swift have the full access to the Webkit/Blink api the same way Javascript does and even more with the patterns from the Chromium renderer (for instance, access to lifetime events, OnDocumentLoaded, OnVisible, OnFirstPaint, etc..).

The offline first comes from the fact that every application runs first as a daemon which provide a gRPC api to the world of the services it offers, where the web-UI application process might also consume this same api from its daemon -process manager or from other applications.

Note that the daemon process who is always running give access to "push messages", so even if no UI app are running things can still work, like syncing into the DB, or even launching a new UI application given some event.. This service process also is pinged back by the web-UI process for every event, so the daemon process can also act when theres a need (when everything is loaded on the UI, do this..)

Also about the article, note that with this solution you will code in pure Swift (not WASM) just like the article is pointing out the web applications that can be built without any javascript.

Other languages like Rust, C++, Python can be added giving the Swift applications are talking to the runtime through a C api. (And i could use some help from other developers who also want this in other languages)

If you want to ship your applications with this, you will get a much better environment than Electron, using much less resources, as things are being shared to every application, and the cool part is the applications can consume one another services (for instance a Mailer application) forming a network, and depending one-another (the magic here is that they are all centrally managed by a core process that is always running)

I can relate to a lot of things he said, but one thing i used to pass through that was to dedicate myself to a bigger goal.

I've started a side project that took me some years (at least 4), a lot of perseverance, because there are a bunch of things that are actually quite boring to implement. (You actually need to create some thick skin to overcome those boring tasks that sometimes can take weeks)

Lost basically all of my friends in the process (i'm a social person actually), with the exception of a few who kind of understood it.

No weekends, working from 12 to 14 hours a day, having to mix and deal with already bigger and established codebases.. its the loneliest job in the world and you need a lot of mental balance and good mood to go through it.

But some special spice that actually helped me going on was that there is a social goal in the project and the idea is to provide a way out of the FAANG centralized and controlled world.

People are mostly unaware of where we are heading it as long they have cool gadgets to play with, and governments might only act when its to late. So while everybody is enchanted with this brave new world, the way the things are heading is actually a pretty dangerous one (and i really hope to be wrong on this).

This was a very special reason that keep me going even in the hardest parts (as for instance when i lost my dad to COVID).

Its not really finished yet as it needs some polish, but giving it needs just a little love, at least now i'm able to do interviews for steady jobs (specially now with more remote ones) and get out of this life of doing freelancing work which sometimes is not very fun.

Anyway, my point being, that maybe there's a need for something else, as in my experience, just intellectual curiosity wont do the trick.. (I`ve had some of those too)

For instance even with burnout (which i kind of postponed to the last moment), i'm motivated to go through it all to see this project have at least the (little) recognition it deserves and i don't care to have recognition myself nor i've done it to become rich, as there were much better and easy to implement ideas to that goal, but i just want to see it "on track", having a way to evolve and become a viable alternative road to another kind of future giving us back the power that is actually ours in the first place.

Just as a matter of research giving your background:

Would you like/consider to use C++ but with the web as the UI/presentation layer?

No javascript, no WebAssembly. Just pure C++ talking nativelly with the web layer..

Im working on something where applications can use the web as api, just as ( and even more than) Javascript can, with other spices added to the whole thing.. and im considering to jump to C++ as the second SDK (the first one already implemented being Swift)

Just to understand, what someone like you would think about this, if you dont mind me asking for this..

Yes, it can even serve to expose the others WebRTC api's that are meant for media, where you can have a initial RPC booststrap interface, that will call your application service, that in turns its programmed by you to use the WebRTC api to stream audio and video from that peer. (or use the datastream if you want)

Good luck with this, I've been reading all the various HN posts on this topic and you seem to be the closest to the holy grail.

Thanks, its nice to hear that after all those years working on it and keep going by understanding how important is for all of us to get out of this rigged game where the cloud computer(+ web clients rulers) FAANG's are controlling where we are heading and a future where we(hackers) have less options technologically, not having enough freedom to program services that can ignore the mothership and create the same sort of services only relying on peers, is a goal being pursuit by them.

I think this would be interesting if the application itself was just a Docker container that could output to a browser, similar to many other local-hosted approaches[1].

What my solution do is that the application have a "service" process that works like daemon to the application, being able to resolve route requests both locally (when over IPC) and remotely (when over RPC).

The routes are meant to serve html or any other content to the UI application (which is also another process akin to the Chrome renderer).

But the fact that every application might have a ever-running process, means that in some cases it can create other kind of services. For instance:

You can create a PostgreSQL wrapper application where you present the the database SQL api over RPC as a service, and manage the PostgreSQL instance with your service process.

The same works for Docker or QEMU for instance. Your application can be docker-based and can be deployed only on Linux.

Your applications (the service and the UI process) are natives by the way, and even the UI application controls the Web layer (rendering, layout, etc) nativelly talking directly to it (as a first SDK enabled over Swift for now).

This architecture will allow you to do those things, and present a way for the users to interact with your solution in a UX that is packed and develop together with the "backend".

With this hole thing you program the frontend and the backend as a whole, in the same programming language.

That's interesting. Are you aware of the absurd-sql project?

No, thanks for pointing that out, will take a look into it.

Also, how do peers find each other?

The idea is to use the torrents as a common shareable resource where given the peers have the same interest in that torrent, lets say the torrents works as a "meta-database" with just enough immutable metadata info, giving the developers of that application a list of peers that will have the same RPC service you designed, with a interface that suits your purpose according to your application goals (a distributed Youtube for instance), and giving you know and designed that API yourself whatever you want from each node, lets say a piece of data, be it a file or a database key-value range, you can ask your API for it, combining the torrent peers and whatever distribution combo you need.

And finally, do you intend to open source it?

I've just did

https://github.com/mumba-org/mumba

It's badly documented given i'm on the final touches before a proper launch, but the "documentation" of the storage layer is planned for today, giving how important it is for the whole thing.

The first thing i've tried to get right was the storage layer, and the capacity to use mutable sqlite databases over torrent (together with files which are simpler given they are meant to be immutable).

Given theres no proper doc yet, i can point out to the source at

https://github.com/mumba-org/mumba/tree/main/lib/storage

where:

https://github.com/mumba-org/mumba/tree/main/lib/storage/bac...

is a modified chrome cache storage layer (which is the real underlying disk storage) that abstract the files and databases storages that from the bit-torrent layer perspective are on the disk.

https://github.com/mumba-org/mumba/blob/main/lib/storage/sto...

is the front end

and the:

https://github.com/mumba-org/mumba/blob/main/lib/storage/tor...

is the main abstraction that may be a "fileset"(collection of files as in torrent) or a dataset/database, which in this case you can get the sqlite db handle from the torrent object and deal with it as normal database.

I've take care to enable a key-value store over the sqlite btree, so both form of databases are possible, a key-value and a normal SQL database.

Key-values are important for the distributed case where you may want the nodes to have partial data and abstract a SQL layer (or whatever) on top of the distributed nodes, which is a better solution for distributed storage.

For the database distribution, a 64k SQLite memory page maps to a torrent piece of the same size, which can be synchronized over other peers that knows what's the root of the merkle tree of the given database is (you can use the RPC layer to coordinate this or use the bit-torrent DHT updating your slot with the new merkle root)

BTW This is what i'm using to distribute the applications, the DHT which points to a "database torrent" which in turn is a index to other files and database torrents. The application owner have write permissions over that particular DHT slot and can change the root merkle anytime the application itself changes (a new version, or some of the assets).

What im finishing right now is exactly the higher level layers that automate this whole thing and make it work "under the hood" without the users or applications distributors need to understand how it is implemented.

Of course the developers will have access to iterating over the peers of a giving torrent to group them over RPC interfaces, and the idea is also to give access to the DHT so solutions that need a access to mutable ever-changing merkle root is also possible (even though the torrent based solution cover most if not all of the ground), but the idea is to give the devs the tools to be creative about their solutions.

(The solution is a whole browser-based UI/web applications platform, that use torrents and the torrent DHT for the p2p storage layer)