Phantom OS

by ectopod • 4 years ago

202 99 4 years ago

mikewarot• 4 years ago

This is an interesting idea, much like KeyKOS in the past, which had a heartbeat that saved the entire system state every N seconds, tagging blocks of memory as it went so it didn't have to be instantaneous. It was demonstrated by simply powering the system off, then back on, without letting the OS know it was about to happen. After about a minute, everything was back up and running, like nothing had happened. (Or so I've read)

Supporting Capabilities is the one thing I'm looking for in a new daily driver. I don't care how rough it is to use, as long as I can compile and fix things, I can tollerate quite a bit. I lived through MS-DOS and everything since.

monocasa• 4 years ago

Interestingly, KeyKOS's successors ended up removing the transparent checkpointing out of the kernel. As I understand it the reasoning was two fold. 1) virtual memory page outs can be implemented cleanly by a user space process, so pushing it out there makes the kernel have a smaller surface area. 2) checkpointing in distributed envs requires some idea of the checkpoint event being propagated to user space since there's all sorts of state that could be on other systems. Rolling back is going to get TCP, etc out of sync. The earlier user space knows it's been rolled back to a checkpoint, the quicker it can recover practically.

blenderdt• 4 years ago

I believe this is related to the Blender file format (.blend). It is just a dump of the program state. That's why saving and loading is extremely fast.

throwaway684936• 4 years ago

I'm glad to see real OS research hasn't stalled. This feels very Plan 9-esque in its boldness and commitment to its ideas. We need to let go of Unixisms.

GekkePrutser• 4 years ago

This has a POSIX layer so it's not free of 'unixisms' by design.

I agree it's a always great idea to explore new ideas from zero without any conceptual baggage. I'm sure that unix-like OSes will still have a lot to offer for some usecases though.

agumonkey• 4 years ago

I too would like to see less unixy OSes more often. I'm happy to see this on HN

brink• 4 years ago

We need to let go of Unixisms.

Why?

convolvatron2• 4 years ago

well for starters some of the machinery is useless and requires constant stepping over.

some design decisions (fork, signals) have really shown themselves to be mistakes as time goes on

for what it does on a server system for example (manage processes), there is a lot of useless genuflecting and papering over problems. remote manageability being a big one

so no, no one is going to die. but we could do a lot better.

azinman22• 4 years ago

https://web.mit.edu/~simsong/www/ugh.pdf

flenserboy• 4 years ago

One simple reason: too similar & then everything gets ported over, & the OS becomes YET Another Annoying OS Because It's Familiar But Feels Wrong. Yes, it's not great to have to make new, basic tools, but having tools that fit the environment is better than shoehorning in tools & demanding that compromises be made in the environment so that the old tools work more easily.

checker6593• 4 years ago

Can't remember who said it, but Unix is a virus.

nine_k• 4 years ago

I would say that in this case, it has.

I remember this OS project starting about 20 years ago. Since then, it haven't shipped much.

The idea behind it is to make the whole OS into basically a giant Java heap. Everything is an object, referring to other objects. It gives you some nice properties, because everything can become a capability which can't be forged. Also, everything can automatically be persisted and restored, Smalltalk-style.

The problem here is, of course, garbage collection. When not just your whole RAM but your disk storage is a mesh of pointers, GC becomes a resource-intensive task.

I wish they pivoted to building an OS for Opteron architecture; there the approach might have a chance to shine.

scroot• 4 years ago

If the computers of the future make use of advances in non-volatile memory, it might change this picture.

checker6593• 4 years ago

OS research hasn't stalled

It most definitely has. But, there are people working on the field regardless.

Here's a very similar project out of Waterloo : https://www.rcs.uwaterloo.ca/pubs/sosp21-aurora.pdf

pjmlp2• 4 years ago

From the looks of USENIX papers it is as alive as always, researchers prefer to focus on distributed computing OSes, as there is where the research money now lives on, the classical desktop is done.

Mobile OSes (which can be turned into desktop like experiences) is where most research activities in managed stacks and improved security are now going on.

productceo• 4 years ago

Cool idea!

mabbo• 4 years ago

Without any intent to take away from how cool this is (seriously, it's awesome) I do wonder what kinds of applications need this sort of capability.

Surviving a reboot is a very specific capability. With no hyperbole intended, who needs it?

Asmod4n2• 4 years ago

macOS does something kinda like it. For when your battery dies you just have to boot the machine and all running apps will sort of start in their previous state.

paxys• 4 years ago

That really depends on how well the app manages its own state and specifically handles restarts. No mainstream OS can restart without running apps even knowing it.

checker6593• 4 years ago

How is that any different from hibernation?

qmmmur• 4 years ago

Running a tiny embeddable computer in a highly contingent environment. Off the top of my head I know someone running such computers to measure biodiversity through audio recordings. They fail all the time due to extreme weather and in one instance a bear pissing on it. I imagine the security could be useful in such an instance

ModernMech2• 4 years ago

I would flip it around and ask "Who wouldn't need this?" Think of it this way, operating systems like Windows have tons of vulnerabilities, which need to be patched, and applying those patches takes a reboot. But people push off those reboots so long because they mean closing all applications and interrupting work, to the point where the entire computing ecosystem is more vulnerable and less updated as a result. Microsoft started scheduling updates in the middle of the night just to find a time that was amenable to users, but that doesn't help when the machine is turned off! So I think that feature, if it were deployed everywhere, would be a net win for everyone, not specific at all.

mgerdts• 4 years ago

But if your application is just restored to its previous state, it is still using the buggy code that was patched on disk. If you page in new code from patched files, the program will not necessarily know that foo() now lives at a different address.

Save/restore across shutdowns is much simpler when the code doesn’t change. When the code changes, you need some form of live patching (ala ksplice) in the mix as well.

There are also concerns when the system interacts with external systems. Suppose the system state was saved just before the million dollar transfer was performed, the transfer was performed, then the cat tripped over the power cord without that being persisted. When the system starts back up, the app may make another million dollar transfer thinking it was the first one.

glandium2• 4 years ago

Windows happily applies updates during the night on laptops even when the lid is closed. Then the next day, you open the lid and bam, nothing of what you left open the previous day is there anymore!

brightball• 4 years ago

Anywhere in regular danger of power failure would certainly benefit from it.

8n4vidtmkvmk• 4 years ago

battery or ups solves that

watmough• 4 years ago

Back in late '80's out of St Andrews and Glasgow, there was a bunch of research into persistent object environments. Here's a paper:

https://archive.cs.st-andrews.ac.uk/papers/download/MBC+88.p...

In short imagine Java or C++ objects, but they can live in a soup in a persistent database.

Imagine if your programming assignments at school, were "open this database then write a client to talk to the object retrieved." That's what we actually had in PS-Algol in 86. Super cool!

jhgb2• 4 years ago

In short imagine Java or C++ objects, but they can live in a soup in a persistent database.

AllegroCache still exists today. So at least Lisp people don't have to imagine that. I wonder if that is one of the reasons why CL people in the 1980s came up with UPDATE-INSTANCE-FOR-REDEFINED-CLASS.

ianai• 4 years ago

Because persistent storage is about as fast as ram. (Not yet but eventually)

zuminator• 4 years ago

As to who would need the OS state surviving reboot, what comes to my mind is things like Mars rovers and other autonomous exploratory vehicles. Currently these use cases are filled by VxWorks[0] among others but a fresh approach might yield some advantages.

[0] https://en.m.wikipedia.org/wiki/VxWorks#Aerospace_and_defens...

rco87862• 4 years ago

Imagine the user experience of closing the lid on your laptop and reopening it a few hours or a few days later, without using any battery life in between.

You can also imagine it being useful in extremely low power solutions where battery life is critical. Space craft, etc

marcellus232• 4 years ago

Imagine the user experience of closing the lid on your laptop and reopening it a few hours or a few days later, without using any battery life in between.

MacBooks and probably other laptops (?) already do this. You might lose a couple percent over a few days.

spacexsucks• 4 years ago

Lenovo S3 sleep state already does this. I have been using this for like 5 years atlesst. systemctl hibernate

hdjjhhvvhga3• 4 years ago

I'm a bit confused. Thhe hibernate function has been available in Windows (and also Linux) for years. In Windows 10 you might need to enable it in Power Options in the Control Panel. I'm using it all the time. It's very useful for long travels for example.

tomjen3• 4 years ago

I basically leave my MacBook on sleep all the time so that it is ready to pick up where I was.

With this I could do the same with a desktop computer without having to lose power.

dgb23• 4 years ago

This is really cool, I felt deeply relaxed and curious at the same time just thinking of the implications.

There’s got to be tons of stuff to dig out and rediscover from early CS R&D. Some of these are more advanced than what we have today, but we seem to be obsessed with patching over existing solutions.

PyWoody• 4 years ago

GitHub looks pretty slow/dying: https://github.com/dzavalishin/phantomuserland

Sirened• 4 years ago

I'm pretty sure it's just one dude working on the project. I remember this popping on the hobbyist OS dev form a while back.

nhumrich• 4 years ago

My first thought is that this computer will be impossible to use. Why? How many bugs exist in computer software, where the solution is to turn your computer off and on again. If power loss doesn't effect state of running programs, then once something is weird, your only option is to reformat the machine. The IT crowd will be out of a job.

themodelplumber• 4 years ago

There will be more IT Coaches.

They can show you the path to successful OS use. But only you can walk it. Mistakes must be accepted, contextualized, and built upon as a path forward... :-)

ThalesX• 4 years ago

As an IT OS Coach, it just irks me when people don't understand how to use their OS properly and blame the 'system'; people forget how the before-times were, when you had an OS thrown on your hands and had to waste precious time learning to use it.

wyldfire• 4 years ago

Don't Android and iOS apps use a persistent database to store much of their data? So in that case the recovery procedure is to do something like "Clear storage"/"Clear cache".

And most of your work laptops probably have a network-mounted user home directory. Swap a laptop and you still get your home dir. You can always selectively purge part or all of your home dir.

Persistence doesn't make these issues unrecoverable, it just makes the process for recovery different. Although admittedly it's less simple than it was in previous generations.

johnisgood3• 4 years ago

We will be out of a job because we will not be able to say "have you tried turning it off and on again?". On a serious note, there are already loads of IT jobs that do not actually have to exist... Soon we will have a job specifically for coming up with the tech stack, picking frameworks and so on. :)

kfrzcode• 4 years ago

This feels like a security nightmare, though, from someone who has no working OS-level knowledge

nextaccountic• 4 years ago

What's the license? Is it open source?

brantonb• 4 years ago

LGPL-3.0

https://github.com/dzavalishin/phantomuserland/blob/master/L...

skybrian2• 4 years ago

Transactions (something like fsync) and code upgrades (something like database migrations or hot-reload in a debugger) seem like the tricky parts and I don't see a discussion of how they'll handle it.

Let's assume you get a stream of security patches from somewhere. How do you apply them without a reboot? What if a data structure needs to be migrated?

During early development, it's easy to just throw away all your data and start over after a redesign, but as with production databases or filesystem implementations, once you're storing real user data, you're not allowed to do that anymore. It's helpful architecturally to have a distinction between on-disk structures (which you're careful about either not changing or migrating) and in-memory structures (which don't) rather than trying to freeze or migrate everything.

zozbot2345• 4 years ago

Transactions can be done "easily" by reusing virtual memory support to trap accesses to memory (read/write/execute) and basically running coherence protocols in software. This is also how distributed transactions can be implemented, which enable you to deploy a bunch of threads running in a single address space across separate nodes in a cluster. It's not all that much harder to implement than checkpoint or restore, which they seem to have already.

bitcharmer• 4 years ago

If I understand this idea correctly, it would have a huge performance impact if every load and store had to be trapped. Or am I missing something here?

vanviegen• 4 years ago

It's helpful architecturally to have a distinction between on-disk structures (which you're careful about either not changing or migrating) and in-memory structures (which don't) rather than trying to freeze or migrate everything.

That's true, but syncing between the database and in memory data structures is tedious. Imagine building an application and never having to worry about storage. Just instantiate and reference objects and you're set. When working with lots of data, the OS would just swap things in on demand. That would save a lot of development time for many types of applications.

But yes, there would be some big challenges regarding application updates and how not to destroy all data with a trivial bug. (A misplaced `customers = []` might be all it takes to permanently wipe all data. Perhaps this idea should be combined with immutable data structures and versioning.)

[Edit] After some more consideration: what I'm talking about feels more like a language feature than something the operating system can/should do.

skybrian2• 4 years ago

Serialization doesn't go away because you still need it for network connections. Data formats useful for sending messages over the network can also be useful for saving things to disk. These formats should be OS and language-independent and have rules for backward compatibility.

Consider how hard it would be to replicate a git repository if it's just a bunch of ad-hoc classes defined in one language.

Protobufs are one design that addresses these concerns. There are others. There is an impedence mismatch, but maybe this would be better addressed through languages that have better support for serialization?

phoe-krk• 4 years ago

For code upgrades, see e.g. multithreaded Common Lisp implementations which achieve this consistently via extremely late binding. Of course, things get a bit more complex if you e.g. need to atomically synchronize upgrading multiple code bodies rather than only one, but that is something that can be done in application logic itself.

unixhero• 4 years ago

And they got Quale built and running. Well played PhantomOS.

dang• 4 years ago

Recent and related:

Phantom OS: Persistent Operating System - https://news.ycombinator.com/item?id=30283864 - Feb 2022 (68 comments)

Also:

Phantom OS, a Russian OS where “everything is an object” - https://news.ycombinator.com/item?id=19672610 - April 2019 (23 comments)

legalcorrection• 4 years ago

The reason why "turn it off and on again" solves most computer issues is that the issues are caused by the system entering unexpected states that weren't properly accounted for at design/coding time. The longer a system runs, the more likely it will end up in such a broken state. So basically, this system is designed to monotonically drift toward being completely broken. How is this an improvement?

denton-scratch• 4 years ago

I suppose there's a procedure of some kind for rebooting from zero, rather than restoring previous state.

I guess I'd like to be able to specify at startup which processes are to be de-hibernated, which are to be restarted from scratch, and which are to remain dead.

Cloudef• 4 years ago

Yeah I stay clear from anything java inspired

Aleksdev• 4 years ago

Very cool project! Refreshing to see some cool non Unix systems.

secondcoming• 4 years ago

On the application code (VM bytecode) level OS shutdown (either manual or caused by failure) is not even 'seen' - applications and their data are 'never die', they continue their work after the next OS boot up as if no shutdown ever happened.

How does this work with timers?

eternityforest• 4 years ago

This is really cool, but at odds with SSDs. They don't like all the write cycling, and at the same time, they can boot fast enough to not need persistence, and are low power enough that people just leave them on all day.

Lack of persistence could almost be seen as a benefit, because as we all know, most problems are solved by a reboot.

But I'm sure somewhere out there is a super niche application for this.

omneity• 4 years ago

Could PhantomOS’s architecture be a great match to XPoint (3d memory like Optane) or eventual memristor memory, potentially achieving instant booting times?

brendi• 4 years ago

Ignoring the actual OS -

the website uses some Google and Yandex services, and doesn't work without JavaScript enabled....

bobuk• 4 years ago

The author of the concept is a pro-government-minded Russian dude who actively supports Russia in this war against Ukraine, Dmitry Zavalishin.

It is sounds not so important, but I think there's no way to build something new for the whole world (like new OS) with a founder who supports Putin.

CPAhem• 4 years ago

A review of Phantos OS from 2009

https://www.theregister.com/2009/02/03/phantom_russian_os/

Phantom OS

Related Stories