Writing my own init with Go

http://www.mustafaak.in/2016/02/08/writing-my-own-init-with-go.html

by CSDude • 10 years ago

149 71 10 years ago

_yy• 10 years ago

Nice project! Even if you believe that systemd is too complicated, it's a great example on how to build an asynchronous, modern init system. Upstart uses some highly questionable tricks (ptrace!) to track processes across forks whereas systemd uses cgroups.

pjmlp4• 10 years ago

Not only that, it is a nice example of using Go in a role that many would use C for, so it might convince a few to try their hands to safer programming languages.

Next step, bare metal runtime! :)

lmm3• 10 years ago

Is there even any concurrency in this case? Wouldn't e.g. OCaml be a better choice?

vezzy-fnord• 10 years ago

cgroups are overkill for process tracking, per se. The Linux kernel has the Netlink proc connector for that purpose. Subscribing to cgroupfs entails a much higher cognitive and resource load in having to deal with a hierarchical resource management subsystem altogether.

Upstart is much more readable even though it has various idiosyncrasies.

digi_owl• 10 years ago

I get the impression that the primary reason for systemd using cgroups was "security". This in that it provided the means of namespace isolating daemons.

This in turn because if you follow Poetterings project history it starts with him creating Pulseaudio to handle moving between sound devices while in use, then crated Avahi to provide a means of Pulseaudio equipped computers to find each other on a network, and then came Systemd based on what he learned about daemon security while writing Avahi.

Sadly i can't shake the feeling that the security in systemd is all too often leaning towards "security theater".

jabagonuts• 10 years ago

The reason that I wrote sleep 3 seconds is that I want to see how the kernel reacts when init dies.

That was my favorite line in the entire post. I love seeing this kind of fearless experimentation.

When I first learned to program, I desperately wanted everything to work and would try very hard not to break things. As I've matured into senior roles, I spend of a lot of effort creating "safe" development and testing environments like the author did here, just so that I can break things and make observations and then throw the environment away when I've learned what I needed to learn. Tools like vagrant and cheap cloud computing have made this easier to do than ever.

agumonkey• 10 years ago

it should be called acquiring information, not 'breaking' things :D

kough• 10 years ago

"Actively exploring modes of failure"

vog2• 10 years ago

@CSDude: Thanks for sharing this very interesting project!

From a technical point of view, what are the reasons you chose Go over alternatives, such as C, C++, OCaml or Rust?

(Just curious, as all those languages have compilers that produce fast, optimized, self-contained binaries.)

CSDude6• 10 years ago

Thank you. I found it easier to develop in Go and I use Go in the work environment heavily. But compared to C/C++, I think having a garbage collector is a must for modern init, and I will write all the other utilities in Go as well for better maintainability.

vog2• 10 years ago

Do you also have some comparison to Rust?

Rust has a very advanced memory lifecycle management. The promising part is that it may give you almost all of the advantages of a garbage collector while not needing an actual garbage collector in most cases. Like RAII in C++, but much more advanced.

I'd be interested whether this actually works well for a nontrivial project such as an init system.

pcwalton3• 10 years ago

Garbage collection isn't a must for a modern init.

Memory safety may arguably be, but I see no reasonable argument that garbage collection (a particular strategy for memory management) is.

SixSigma4• 10 years ago

Linux is mostly perfect as it is

Your critical thinking has not been engaged. Linux is a useful bit of code but "perfect" is not what it is known as everywhere.

My side of the room like this little ditty

Linux, by amateurs, for amateurs -- Dave Presotto [1]

i’ve wondered whether Linux sysfs should be called syphilis -- Charles Forsyth [2]

Or if you like your detractors with a bit more fame

Unix has retarded OS research by 10 years and linux has retarded it by 20. -- Dennis Ritchie as quoted by by Boyd Roberts in 9fans.

I won't go on

[1] http://research.google.com/pubs/author4927.html

[2] http://www.terzarima.net/

PopsiclePete• 10 years ago

So what does an OS purist like yourself run? BeOS? Plan9? All OS's suck. They're just horrible. But so is everything else. It's all flawed in one way or another. Windows is flawed, OS X is flawed, our tools are flawed, our languages, our editors - everything. If you want perfect, you have to throw away your computer and somehow live inside Knuth's books.

tronje2• 10 years ago

If you want perfect, you have to throw away your computer and somehow live inside Knuth's books.

Either that or wait for GNU/Hurd.

SixSigma4• 10 years ago

Try reading what I said.

I said "Linux isn't perfect, like you think it is"

sotaan2• 10 years ago

If you improve the perf/functionalities of init using Go, I will give this language a go right after. It seems promising tho.

sotaan2• 10 years ago

Is that possible to recode init in any other language using your technique? For instance can I try it in Ruby?

tronje2• 10 years ago

Since init is the first process that runs, you can't use an interpreted language like ruby (since the interpreter would have to run first, which is impossible if your program itself is init). Maybe there's a Ruby-compiler out there somewhere, but as-is Ruby would be impossible.

falcolas2• 10 years ago

The author might look at how a minimal (yet fully functional) init works:

http://git.suckless.org/sinit/tree/ (look at config.def.h and sinit.c)

and build up from there. Given how minimal sinit is, it's a great place to start your functionality from, and see the power (and limitations) of a basic SystemV init system is.

CSDude6• 10 years ago

Wow, this is great, thanks!

teh_klev• 10 years ago

Part two is now up on his website:

http://www.mustafaak.in/2016/02/09/forking-process-in-myinit...

daveloyall• 10 years ago

@CSDude, I recommend putting a link to part 2 on the part 1 page.

CSDude6• 10 years ago

Done! Thanks.

zettahash• 10 years ago

Using Go even earlier in the boot stage in an initramfs (as a replacement for tools like dracut) crossed my mind recently. Go's statically linked binaries and fast builds make this particularly appealing to me. With (a lot of) work you could forego busybox entirely.

loudmax• 10 years ago

Good point about static linking. Though Go's big fat binaries count against it as a busybox replacement in areas where storage space is a premium.

chubot6• 10 years ago

NOTE: You can't write an init without threads in Go, as you can in C. (I tried awhile back)

At least you can't with its portable interfaces such as exec.Command() and cmd.Wait():

http://www.mustafaak.in/2016/02/09/forking-process-in-myinit...

You end burning a thread per process, which is pretty lame for an init. The wait() call blocks a goroutine AND OS thread, so the go runtime will have to start a new thread for the next wait().

Whereas in C you can do the entire thing from a simple thread (I call it "async processes"). You set SIGCHLD handler and get async exit notifications, and then process those in a single-threaded loop. I'm pretty sure there is a problem doing this in Go (besides resorting to the syscall module)... I think it has to do with the fact that Go turns signals into messages on a channel, but I don't remember right now...

EDIT: Now I remember. The problem with using the syscall module is that Go does not export Fork and Exec. It exports ForkExec, because it needs to ensure they run on the same thread.

https://golang.org/pkg/syscall/#ForkExec

This severely limits the usefulness of your init program, because in Unix there are all sorts of important things that happen in between. It's how you set the child process state. You basically can't do anything with file descriptors or containers at all with this model.

pcwalton3• 10 years ago

Signal handlers are pretty much impossible to write directly in a managed language (at least without heavy contortions from the runtime), because you could get a signal during a non-GC-safe point and then deadlock or corrupt your program. For example, malloc often takes locks internally; if you get a signal during malloc with that lock held and then try to malloc in your signal handler you can deadlock. Managed languages like Go treat allocation as an implementation detail rather than something the programmer has to explicitly invoke, making them basically incompatible with signal handlers.

(BTW, this is mostly a problem with signal handlers as a concept, not with Go.)

The problem with using the syscall module is that Go does not export Fork and Exec. It exports ForkExec, because it needs to ensure they run on the same thread.

And because you can't safely allocate after a fork in managed languages, because fork drops your other threads, leading to the same problems above. :)

chubot6• 10 years ago

So the way Python does it is that it doesn't call back into the interpreter loop from the signal handler. That would be a really bad idea as you note, because the interpreter can call malloc and do all sorts of other stuff.

It queues your Python signal handling function, returns from the signal handler, and then runs it on the main loop at the next tick.

As far as I know this works fine. I wrote the same program in Python and it seems to work just fine, and have more functionality than the Go version. I guess one problem is that if you do a long blocking syscall on a tick in the main loop, your signal handler can be delayed indefinitely. There might be some hacks around this in the Python interpreter, but I don't recall offhand.

pjmlp4• 10 years ago

Writing any code in a signal handler besides setting a flag is a very bad idea, specially when writing code across UNIX compliant OSes.

Unless it changed, POSIX gives no guarantees of stack size and function calls re-entrancy guarantees depend a lot on the OS and compiler versions.

Hence the best approach in managed system languages is just to mark a signal as active and re-trigger it outside the handler when control gets back to user space.

sebcat2• 10 years ago

I don't understand. How is using ForkExec, Wait4 forcing you to use threads? What do you mean with "You basically can't do anything with file descriptors or containers at all with this model."? It is possible to pass a ProcAddr containing a SysProcAddr to ForkExec with fds, chroot, namespaces, &c.

Attached simple demo code of what I mean with ForkExec, Wait4.

EDIT: to clarify, the demo code is not intended to work like init, just to spawn a couple of processes and wait for them to terminate.

        package main

        import (
            "log"
            "syscall"
        )

        func main() {
            cmds := [][]string{
                {"/usr/bin/sleep", "4"},
                {"/usr/bin/sleep", "5"},
                {"/usr/bin/sleep", "1"},
                {"/usr/bin/sleep", "2"},
            }

            for _, cmd := range cmds {
                if len(cmd) > 0 {
                    if _, err := syscall.ForkExec(cmd[0], cmd, nil); err != nil {
                        log.Fatal(err)
                    }
                }
            }

            for nwaited := 0; nwaited < len(cmds); nwaited++ {
                var w syscall.WaitStatus
                pid, err := syscall.Wait4(-1, &w, 0, nil)
                if err != nil {
                    log.Println(err)
                } else {
                    log.Println("pid", pid, "exited", w.Exited(), "exit status", w.ExitStatus())
                }
            }
        }

chubot6• 10 years ago

OK, I see that there is a lot of stuff like chroot and UIDs in SysProcAttr [1]. I think they have added a bunch since I last tried this.

But the point still stands... you get a subset of the OS API. Everything has to be "canned" into that struct and you can't execute arbitrary code between fork and exec.

Especially with containers, this is a never-ending target... They apparently have user namespaces, but what about network namespaces or IPC namespaces?

[1] https://golang.org/src/syscall/exec_linux.go

Now, it might be good enough for your application, and it probably IS good enough for everything except a full-fledged production quality init system. For that case I would be worried about running into limitations.

falcolas2• 10 years ago

So, don't do process management inside init. Spawn a child and check on it after every "wait" to handle the process management. Less opportunity for the kernel panic, less for init to handle (fewer bugs), greater flexibility.

vetrom• 10 years ago

wait() doesn't need to block at all actually, you can use WNOHANG.

chubot6• 10 years ago

Yeah but then you're polling, which isn't good as mentioned elsewhere).

tkinom• 10 years ago

Writing init in Go probably have size issue compare to typical C implementation. Last golang helloworld I built was 1.4MB. The /sbin/init in my ubuntu dist is only 194k. Busybox version of init optimized for size/features can be much smaller.

Tough to use it in OpenWRT, DDWrt type of embedded system where system might only have a few megabytes of RAM/ROM.

CSDude6• 10 years ago

Yeah, you are right. Go is too heavy weight. But the nice thing in Go 1.4+, you can install go std and dynamically link your applications. I will do it in the next blog posts to save some space, but yeah, I would still need some huge lib.

joshwget• 10 years ago

Although very different than a traditional Linux distribution, the init system for RancherOS might be worth taking a look at. It's written entirely in Go as well. https://github.com/rancher/os/tree/master/init

mmarx2• 10 years ago

The kernel already supports acquiring a DHCP lease via the “ip=dhcp” boot option. Why does this need to be a feature of init?

pjmlp4• 10 years ago

Because init is common to several UNIXes and ip=dhcp is a Linux specific feature?

mmarx2• 10 years ago

Because init is common to several UNIXes and ip=dhcp is a Linux specific feature?

And yet the author seems to be comfortable using those linux-specific features:

“Also, the kernel paremeters must have the rw flag or you have to remount the /dev/sda1 as rw, but changing kernel parameters are much more easier.” (from the follow-up post).

CSDude6• 10 years ago

I did not know we could have ip=dhcp option, but I will need to create a network manager now or later, but for trying out some stuff, I can use that flag. Also, I will not bundle services to init, it will be just forking, signaling and waiting. All the other utitilies will be small independent code exercises for me.

Writing my own init with Go

Related Stories