nderjung

Hi, Alex from Unikraft here. One clarification: Browser-Use didn't move away from Unikraft because of any limitation around browser startup times, snapshotting, scale-to-zero, or browser-level autoscaling. Those capabilities worked well and were a key reason they adopted the platform in the first place.

The challenge they encountered was at a different layer: horizontally scaling the underlying EC2 infrastructure. At the time, native EC2 fleet autoscaling wasn't yet supported by the platform, so they chose to take ownership of that part of the stack and build directly on Firecracker.

It's also worth noting that Unikraft is actively working on transparent infrastructure autoscaling (with live migration), so the gap they encountered is being addressed. The article's title may give the impression that unikernels were the bottleneck (they weren’t, and our platform transparently support Linux VMs as well), when in reality the decision was driven by infrastructure orchestration requirements rather than browser runtime capabilities.

As an aside, we love Browser-Use <3 and we still work together closely!

Howdy! We are hard at work at improving the DX, and as a result we've been working on a brand new CLI. We haven't made any announcements yet, but it's already open-source for early adopts if you'd like to give it a try!

https://github.com/unikraft/cli

Feedback is very much appreciated, we're listening! :)

Hey! Co-founder of Unikraft here.

Unikraft aims to offer a Linux-compatible environment (so it feels familiar) with the ability to strip out unnecessary internal components in order to improve both boot-time/runtime performance and operational security.

Why would you need a memory allocator and garbage collector if you serve static content? Why would you need a scheduler if your app is run-to-completion?

Linux gives you the safety-net of generality and if you want to do anything remotely performant, you by-pass/hack it altogether.

In the article, Unikraft cold-boots in 150ms in an emulated environment (TCG). If it was running natively with virtualization hardware extensions, it can be even shorter, and without the need for snapshots which means you don't need to store this separately either.

Which OS and device are you using?

This project operates by first building the webassembly module into an ELF binary and then statically linking this against the Mewz library OS in order to become a unikernel binary.

It looks still very early days in terms of supporting all libc/syscalls as well as support for different platforms.

I wonder how this compares in terms of performance to https://unikraft.org which is able to do the same but is a more establish libOS/unikernel project (disclosure I am a maintainer of Unikraft :-)) since this is written in Zig and Unikraft is written in C.

Hi!

Thanks for the feedback!

Good point on the Kraftfile syntax and as you've seen, there is a reference document. We can make sure to place a link to this in all guides.

The term "KraftCloud Specification" is in fact incorrect, this should just be "`Kraftfile` specification".

We are working on providing a "Unikraft Hub". Since unikraft.org is a public OCI registry and it works through many popular tools like `crane` or `regtool` as well as our native `kraft` CLI tool; you can actually view it locally by running `kraft pkg ls --all --apps --remote`.

Thanks again for the feedback! We'll get on to updating the docs to make these bits clearer!

We ended up doing this over at https://unikraft.io :-)

Correct -- and you can run multiple kernels on the machine with virtualization extensions. Even Docker Desktop does this. You'd do this for _real_ isolation purposes.

Containers are perfect for build environments and for creating the root filesystem. The issue is that the kernel these days are super bulky and are intended for multi-user, multi-process environments. Running a container runtime on top just makes it worse when you're looking for "isolation".

This paper argues that when you build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against necessary bits of code to execute _as_ a VM, then you'll get better performance than a container and you'll get that isolation.

This is in fact true based on performance studies, the follow up paper to this shows so: https://arxiv.org/pdf/2104.12721

(Disclosure, co-author of the linked paper.)

We ended up taking this to real workloads if you want to see it in action: https://unikraft.io/

If you're looking for an alternative way to run traefik, we support this out-of-the-box on https://kraft.cloud -- A platform dedicated to running ultra-lightweight VMs based on Dockerfiles, with millisecond cold start times (96ms for Traefik), scale-to-zero, autoscale.

Check it out in our docs: https://docs.kraft.cloud/guides/traefik/

It's also possible to start traefik and other services together using Compose files: https://docs.kraft.cloud/guides/features/compose/

Hi, author here,

You can, and we've written about the differences also, here [0].

The summary is that there is still a performance hit for boot time (100s of milliseconds to seconds vs. 1 to 100 of milliseconds for Unikraft); performance hit at runtime (even removing the syscall boundaries has a less-than-ideal impact based on previous studies); and, there's extra bloat from the image size itself (the image is still at the very least 30MB+ vs. as little as 100KB for Unikraft) which affects startup time and cost of storage + transport.

[0] https://unikraft.io/blog/ukl-vs-unikraft

In fact gVisor is the opposite, it injects more guard instructions between the application and the kernel across the syscall layer in order to make stronger security guarantees. These additional guards slow the application even further by however long it takes to perform necessary permission checks.

It is not necessary to have such checks in a unikernel because the kernel inherently trusts the application because together they were constructed in the same pipeline. The hardware then protects the two together.

Most people don't run metal today

We are seeing an increasing trend towards On-prem/Cloud-prem/Co-los[0], mainly due to cost and reduced complexity. Inversely, most smaller companies (1-10 emp) who use hyperscalers do not use their metal offerings, because of cost. They wish to scale with demand which metal cannot provide. Using EKS and other similar services have the benefit of being familiar and elastic, but are in fact slow and soon become quite expensive[1].

[0] https://www.gartner.com/en/newsroom/press-releases/2023-05-1...

[1] https://a16z.com/the-cost-of-cloud-a-trillion-dollar-paradox...

How many people know unikernels?

This has been a goal of Unikraft for a long time, to make using unikernels simple and familiar to use (in fact, transparent). This is why we use OCI images as the root filesystem; why it's possible to start unikernels through Docker; why we have several types of Kubernetes integrations.

How do you debug a running app?

For one, you can attach a gdb server and step through both application code and kernel code together. Secondly, at Unikraft at least, we are introducing a virtual shell that allows you to introspect the filesystem, main threads, see system stats, etc.

Stripped down Linux distros reduces attack surface

This is may reduce the attack surface, but one bad-actor application can still take down the host and all the other containers since they are still process (software) isolated. With unikernels you get hardware-level isolation AND, interestingly, the performance thanks to the lack of strong syscall boundaries.

Unikernels increase complexity

Give us a chance and try out one of our examples :-)

https://github.com/unikraft/catalog/tree/main/examples

In fact this is not true with Linux. We did a study of this[0] and found that there are too many interdependencies between Linux kernel subsystems. The smallest, usable Linux kernel we test with sometimes is around 30MB.

In comparison, compiling NGINX together with Unikraft results in a kernel size of under 2MB. This means that there is 28MB of kernel we didn't need or want.

[0] https://arxiv.org/abs/2104.12721 (See Figure 1)

Hi, [disclaimer, I am a project maintainer].

The key difference between unikernels (and library OSes) vs. other kernels is that each kernel is different with a unikernel. Your application goes through a process of specialization, where you can decide exactly the necessary features (e.g. libraries, syscalls, memory allocation, scheduler, etc.) are necessary or desired for your application. Additionally, other kernels are typically multi-process whereas a unikernel is inherently single-process since it does not support syscalls like fork. (That said, at Unikraft, we are working on supporting fork, eta. late April).

Additionally, before Unikraft, one of the major trade-offs was developer time and effort to get your application into a unikernel. Since Unikraft has been actively developed for several years by ~100+ people, it has become much more stable, feature complete and easier to use; ultimately making unikernels easier to approach.

Today, in-addition to multi-process applications, the trade-off now really depends on your usecase and so selecting whether you wish to deploy your application as a unikernel or on top of a monolithic kernel or microkernel. For example, developer environments which require many subsystems, multiple programs, etc., where you can't know everything ahead-of-time, are still best suited for multi-user, monolithic kernels that can run most workloads.

Another thing to note is that unikernels are designed to be immutable and are often created in a CI/CD context. This means that they are single-purpose and if you wish to change them, you need to re-build (or re-package).

Hi, Alex here from Unikraft, that's right!

You can find more information about Unikraft here:

- https://unikraft.io -- The Unikraft Company

- https://unikraft.org -- The open-source project

- https://kraft.cloud -- The Millisecond Cloud Platform based on Unikraft

Thanks and, yes it does!

We have a whole section in our documentation's guides section on example usecases with database:

https://docs.kraft.cloud/guides/#databases

They work exactly as you suggest, they scale-to-zero when no requests are made and data can be safely stored/transactioned in persistent volumes on the platform too.

Volumes are persistent, block devices which you mount to a path and are initialized and de-initialized by the kernel which we boot up and shut down, respectively, based on autoscale/scale-to-zero configuration. This means that if your application uses said volume it will be able to access it during its online-lifetime. That said, because they are persistent, it means that they are not as ephemeral (or "scale-to-zero") as the instance itself on KraftCloud.

Because KraftCloud leverages Unikraft, a unikernel library OS project, we are single-process. Postgres is multi-process since it uses the fork() syscall. If you require Postgres, we can set it up for you as a managed service. That said also, we are currently working on supporting fork() and this should become available in the next few weeks/end of April.

Re: bring your own DB: yes. We are setting up metros in many popular hyperscalers and IaaS vendors and DC zones. This means there should be no egress fees and you'll have data locality.

For serious deployments, be close to your data. If you get in contact (alex@unikraft.io), we can discuss further how we are approaching this.

And yes, we are working on cron, this will translate to setting up an instance to start at a regular interval and not auto-restart it (so it scales-to-zero). :-)

Hi,

1. Yes. We're essentially a baremetal solution. Either we run it on your behalf (this is kraft.cloud), or we deliver an ISO or AMI and this can be installed on-prem/cloud-prem. This latter solution is our enterprise offering.

2. eBPF used in this way, I feel, has become a symptom of the problem that we're trying to solve at Unikraft: the bloated Linux- (and cloud native-) stack. In the end, you're adding more tools or doing more tricks to try and do computationally less since performance and running faster is about performing less operations in the critical-path. Unikraft approaches the problem space differently: bottom up (app first, then dependencies, then select OS libraries/primitives based on required syscalls) as opposed to top-down (taking away from Linux, its distros, removing functionality, making short-cuts, etc.).

Definitely! We plan on having a pay-per-millisecond pricing model as well.

There's an initial free tier that, but once limits are reached, we will switch to pay-as-you-go plan. Our next offering ("developer") is still work-in-progress, and we're setting it initially for our early customers with much larger workloads who need autoscale more than they need scale-to-zero.

The biggest take away from this is the LAM-over-email which was hidden away in one of the demos. This has a lot more potential (esp. in a business context) in my opinion than the phone itself.

Being able to directly email an autonomous agent or even CC one later into a conversation with instructions can be a game-changer. Not only would the replies be faster but the result could be cheaper compared to a human equivalent.

One use case which comes to mind is, which was also demo'd, is making trip arrangements. In the demo, the spoken request to book a trip to London was long and precise; no way I would get that right the first time on any push-to-talk device. At my company, we use a travel agent for business trips which we email and they book everything and send over receipts and tickets. I could just as well be emailing a LLM/LAM.

There are essentially three ways to put together a unikernel:

1. Minimizing an existing general-purpose OS

2. By-passing the OS

3. Starting from scratch

You can read more in detail about this here from Unikraft's documentation[0].

[0] https://unikraft.org/docs/concepts/design-principles#approac...

Hi! I'm Alex, I'm a maintainer of Unikraft, AMA! :)

Hi, we at Unikraft are hiring for this role exactly!

We’re a “deep-tech” early-stage startup looking for someone to join our team and lead our frontend efforts.

You can read more about the role at:

https://unikraft.io/docs/jobs/20230403-senior-frontend-dev

We put a lot of effort and consideration into the architecture of Unikraft[0][1], its elegance towards modularity and abstraction is the reason why I joined the team to help develop it. :)

[0] https://unikraft.org/

[1] https://github.com/unikraft/unikraft

I think it can benefit both parties -- underprovisioning for a smaller bill and then increasing when demands need it so as to prevent degregation in QoS. That said, if load increases exponentially high it can induce high costs. Some cloud providers do not offer memory (or just any resource) ballooning, like AWS, so there you will experience the problem you have discussed about over-provisioning. However, we aim to alleviate some of these problems with the uniqueness of Unikernels with our soon-to-be-released Cloud Platform at https://unikraft.io. Features like memory ballooning (with hard upper limits) and other features like deep in-kernel monitoring to understand application performance.

Yes you're right, it's a simple network-like protocol allowing you to mount a path on the host OS to the Unikraft unikernel VM similar to a container volume. In addition, Unikraft's abstract APIs[0] allow for more block devices such as EXT{2..4}, etc. which you mount in a similar way. Alternatively, you can put your filesystem into a CPIO format and mount it as initram and load it into RAM (great for performance and read-only file systems, like webservers).

[0] https://unikraft.org/docs/concepts/architecture/

It really depends on your usecase here. If the many application processes rely on the same OS libraries, versioned language runtimes (e.g. same python version), kernel version, etc. AND you trust the OS then it may make sense. However, unikernels offer the lightweightness of a container process with the security of a VM. In addition to this, memory ballooning[0][1] and other resource-elastic features are available to VMs too: allowng you to under-provision them and then later increase resources when load demands it.

Unikraft unikernels can also be managed using the same orchestration tools as containers, check out the talk at CNCF[2].

[0] https://github.com/unikraft/unikraft/pull/219

[1] https://pmhahn.github.io/virtio-balloon/

[2] https://www.youtube.com/watch?v=cV-xawN9_cg

It works both ways with Unikraft, either bring an existing application and let it run with the added performance/security (and think it's on Linux) or write your application with our performance-oriented APIs[0][1].

[0] https://unikraft.org/docs/concepts/architecture/

[1] https://usoc21.unikraft.org/docs/sessions/10-high-performanc...

We're working hard on adding SMP support to Unikraft which is planned for the next release. The PR you have linked has all the details about the on-going work!