A Look at PureDarwin – An OS Based on the Open Source Core of macOS

https://www.jamieweb.net/blog/a-look-at-puredarwin/

by jamieweb • 7 years ago

244 104 7 years ago

Are there are Darwin hackers lurking on HN? I would pay good money to get a brief summary of the kernel for kernel hackers such as myself who only know how things work in Linux. What are the big architectural differences or cool features not in Linux? Also, how hard is it to upstream changes/get drivers in mainline?

wrs• 7 years ago

Amit Singh’s excellent book MacOS Internals: A Systems Approach [1] is a bit outdated now but still recommended if you’re interested in the workings of a non-Linux commercial OS.

[1] https://www.amazon.com/Mac-OS-Internals-Approach-paperback/d...

monocasa• 7 years ago

Jonathan Levin's *OS Internals volumes are an updated version of the same info, and wonderful.

loeg2• 7 years ago

The Design and Implementation of the FreeBSD Operating System, 2nd ed., in the same category.

https://www.amazon.com/dp/0321968972

saagarjha7• 7 years ago

Not a Darwin hacker, but:

What are the big architectural differences or cool features not in Linux?

Darwin is based on XNU from NeXT, which itself is comprised of OSFMK Mach microkernel but with many of the things pulled into the kernel itself for performance. A lot of the stuff with Mach tasks/ports/messages is still there.

Also, how hard is it to upstream changes/get drivers in mainline?

You can write your own using IOKit but they're not going to be in the mainline tree, because Apple.

loeg2• 7 years ago

Darwin is based on XNU from NeXT, which itself is comprised of OSFMK Mach microkernel but with many of the things pulled into the kernel itself for performance.

And amusingly, the syscall interface from FreeBSD 4 or so.

pmjordan• 7 years ago

IOKit is actually a considerable chunk of the kernel, particularly if you count the various "families", some of which remain open source. I'd say it's Apple's main contribution to the kernel at the architectural level. It's implemented unlike anything in BSD or Linux, as it uses C++ on the kernel side. (The user space side, IOKitUser, is written in C.) These days even Apple uses it for non-device-related kernel services - you can see what the iokit registry looks like on your system using the "ioreg" command or the IORegistryExplorer or IOJones GUI tools.

twunde• 7 years ago

Darwin is actually a BSD variant, so it does tend to absorb some features from the rest of the bsds such as dtrace. Darwin is nigh-impossible to get patches into as an outsider ( I vaguely remember a blog post or maybe a talk about the process). The rest of the BSDs are fairly easy to get patches into upstream/mainline.

sigjuice• 7 years ago

FreeBSD got dtrace in 2009 (FreeBSD 7.1). Mac OS X 10.5 already had dtrace in 2007.

pnako4• 7 years ago

Darwin is not a thing. It's just Apple dumping the code to satisfy the licensing requirements.

ronsor• 7 years ago

Except it's BSD licensed and not necessary to share the code.

saagarjha7• 7 years ago

I wonder if GNUstep works on this…

runjake2• 7 years ago

It does, insofar as GNUstep really works.

winrid• 7 years ago

How doesn't it?

Wowfunhappy4• 7 years ago

If you don’t mind the fact that they’re closed source, could the original Apple kexts be used for networking?

strangemonad• 7 years ago

You’d be constrained by making sure the specific KPI is compatible and additionally that libsa is properly implemented so that it can patch the vtable to dynamically adapt to ABI differences

nutjob22• 7 years ago

No need, you should be able to use Hackintosh networking kexts which are open source.

Wowfunhappy4• 7 years ago

Oh, good point! Although, that wouldn’t have helped the article author running VMWare, as the specific kext he needed was Apple-provided and closed source.

orionblastar• 7 years ago

I am glad someone is keeping up with Darwin and making a version one does not need to compile to be able to use.

I wonder if it can run OSX command line programs?

jki2752• 7 years ago

If you can run it, and they don't link to cocoa, it should be able to do that just fine. Apple used to distribute a Darwin iso, I remember running it in the mid-2000s on an older Mac mini.

Aloha3• 7 years ago

Why would I want to use Darwin?

macOS has some UI advantages, its the largest reason I use it - but without cocoa, and all of the core foundations that make OSX, OSX, I just dont see the point.

stjohnswarts• 7 years ago

Yep might as well just use one of the BSD linuxii

dfox• 7 years ago

For me: you don't want to. The underlying Darwin/XNU layer is probably the most quirky (or depending on your PoV outright broken) unix-ish system you will see in general use today.

Aloha3• 7 years ago

that was the subtext I didn't, I know XNU is not super performant.

skissane6• 7 years ago

z/OS is a much weirder Unix than macOS, although it depends on how you define "general use". But, there are a lot of enterprises out there which still run IBM mainframes, even if they often aren't keen on advertising that fact, and a huge lot of stuff in z/OS now depends on the Unix subsystem, in particular anything written in Java (which IBM has been heavily encouraging as a development language on mainframes, resulting in a lot of hybrid apps which are a mix of newish Java code and ancient COBOL/assembler/etc working together.)

(Some people will question whether z/OS really counts as a "Unix", but like macOS, it is actually a certified Unix, and as such has greater official claim to be Unix than FreeBSD or most Linux does.)

(Addendum: I say "most Linux", because while the vast majority of Linux distributions are not certified Unix, some have been. Right now, Huawei EulerOS is a certified Unix based on a Linux distribution. Inspur K-UX is another Linux distribution which used to have Unix certification, although the certification has expired–I guess they failed to pay the renewal fees.)

Hackbraten2• 7 years ago

Care to elaborate? Which aspects of XNU are you referring to?

derefr3• 7 years ago

Maybe not you personally, but one of the long-term informal goals of PureDarwin is to have an open-source environment you could run XCode CI builds on.

saagarjha7• 7 years ago

But most Mac apps will link against Cocoa…

Aloha3• 7 years ago

Why is a booting XNU helpful for me to build macOS or iOS applications on?

unixhero• 7 years ago

If you don't feel an urge to try it, then it's not for you.

faissaloo• 7 years ago

GNUstep does reimplement alot of Cocoa stuff

MisterTea• 7 years ago

Why would I want to eat vanilla ice cream?

chocolate ice cream has some taste advantages, it's the largest reason I eat it - but without cocoa, and all of the core flavor that makes chocolate, chocolate, I just don't see the point. (like your post)

ngcc_hk2• 7 years ago

Can it run nvidia card Sk my old mac can be used; windows can run it but it optimised.

DavidPiper2• 7 years ago

This was one of my hobbies at University (learning how it works, not contributing to it, to be clear). This SuperUser question was basically the culmination of my effort before I ran out of spare time, found a job, etc.

https://superuser.com/questions/546151/is-it-possible-to-com...

I'm super excited that people are still working on this! I miss having the time to go down rabbit holes, but for those that are there's are people out there watching!

DavidPiper2• 7 years ago

A huge kudos to Jonathan Levin too! It looks like the full set of new *OS Internals books are now available (last I checked only two volumes were available) -

http://newosxbook.com

They are the spiritual successors to Amit Singh's Mac OS X Internals: A Systems Approach (http://osxbook.com).

codetrotter5• 7 years ago

While we are on the subject of macOS internals.

Do you know where I can read about what it would take to replace WindowManager and the rest of the desktop environment on macOS?

I would like to make my own implementation of the desktop environment, including replacing the top bar with the menus and stuff, and replacing the dock with a dock of my own, as well as replace spotlight search and Finder. Furthermore to also replace the open file dialog that is called to from programs when they open files.

mukundmr• 7 years ago

Apple's Open Source repository has the current version of Catalina's core. https://opensource.apple.com/release/macos-10145.html

Why is PureDarwin on an older version? Lack of volunteers? Too many hacks required?

dschuler• 7 years ago

10.14.5 is Mojave, and we're still waiting for the latest 10.14.6 Mojave source. Their open source repo seems to be about 6 months behind at any time.

jonathanstrange2• 7 years ago

Isn't that a license violation?

Wowfunhappy4• 7 years ago

I have to wonder, why does Apple even make Darwin open source?

It doesn’t seem like it’s of much use to Apple—there isn't some large community of outside contributors submitting pull requests. So why do they bother?

mnem2• 7 years ago

It’s quite a good marketing line for the company.

saagarjha7• 7 years ago

How so? Their code isn’t up-to-date at all.

s_dev• 7 years ago

Licence compliance -- they took the Unix kernel and don't really want to give back but having now taken the kernel it now has obligations under that licence to make it available for others.

Redoubts• 7 years ago

Interesting. Which license are you referring to?

Wowfunhappy4• 7 years ago

Which license? I'm not aware of any obligations to release Darwin's source on Apple's part—hence how they're able to wait half a year before releasing the source of new versions.

jjoonathan• 7 years ago

One time I used Darwin open source to make a printf-sprinkled version of the dynamic linker to track down a bug that was causing our app to not start on about 1/3 of our install base. There were no error messages or console logs and the logs produced by feeding dyld environment variables were not helpful, so the ability to build a custom dyld was instrumental.

The problem wound up being that we leaked a sticky bit directory into part of our install, the installer inspector hid this from us, and apple changed dyld's behavior in a patch release to silently bail if it saw a directory with sticky bit set.

epaulson• 7 years ago

I wish the PureDarwin folks would prioritize clear build instructions. It'd be really great to see something like a "Linux From Scratch" or process to build a bootable disk image using only what's in their git repos. (Maybe it exists but if so it doesn't show up/is linked to from any of the obvious pages of their wiki)

That seems like a necessary first step towards eventually getting a bootable ISO that can install a PureDarwin system on a blank machine, physical or virtual.

rnd0• 7 years ago

I'm not really sure how useful it is for most of us -but it doesn't have to be. Not all operating systems are general-purpose.

I'm glad to see it continues to be maintained and published though I share the frustration other's have mentioned about how incomplete it is. There is no "more", no "less", no "halt", no "shutdown" -those are just a few of the "wtf" finds that pop up on casual examination.

A Look at PureDarwin – An OS Based on the Open Source Core of macOS

Related Stories