Linus says Linux has become 'bloated and huge'

http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/

by nreece • 17 years ago

125 85 17 years ago

theregister.co.uk

staunch• 17 years ago

Out of context link bait drivel.

WalterGR• 17 years ago

How so?

> "Uh, I'd love to say we have a plan," Torvalds replied to applause and chuckles from the audience. "I mean, sometimes it's a bit sad that we are definitely not the streamlined, small, hyper-efficient kernel that I envisioned 15 years ago...The kernel is huge and bloated, and our icache footprint is scary.

paddy_m• 17 years ago

Does anyone know what icache is? instruction cache?

stcredzero• 17 years ago

...Windows usually gets the bad wrap for bloat...

At the very least, badly edited.

fnid• 17 years ago

I think it just shows that open source isn't immune to the same kinds of problems that plague commercial software. As the market for a product expands, so too do the features people require for the software to work within an ecosystem of other software. More hardware to support requires more code, bigger systems.

ramy_d• 17 years ago

indeed, it will be interesting to see how this issue gets tackled, if at all.

loup-vaillant• 17 years ago

Off topic: You wrote "commercial", while the correct term was "proprietary". You're not the only one. If I recall correctly, "Open source" was designed specifically to avoid this mistake. Apparently, that failed.

skwiddor6• 17 years ago

My operating system is open source but I bought it. What kind of software is it?

foole• 17 years ago

scamware? ;)

TomOfTTB• 17 years ago

I think this shows the amount of bloat in an Open Source package increases with how widely used the project becomes. As an article posted a couple days ago on HN pointed out about 75% of Linux development is done by corporations these days. Those corporations often times create the same "good enough" code that most proprietary developers do.

Now the theory behind Open Source is that developers around the world can see bloat and fix it. But with very few exceptions the corporate developers aren't going to make their code more efficient. So you have 75% of developers creating bloated code and at best 25% left to fix it.

Meaning the amount of bloated code easily overwhelms those volunteers who are willing and able to fix it.

shin_lao• 17 years ago

The only important factor on code quality is the quality of the people working on it. Open source or proprietary doesn't change anything.

Gibbon• 17 years ago

Roughly 11.5 million lines of code for the kernel.. That's way more than I would have imagined.. 5k lines a day are added and 2k are modified. (http://www.itworldcanada.com/a/IT-Workplace/68f20d4f-8ab9-40...)

IMHO it doesn't really jive with UNIX principles to have such a monolithic kernel.

mahmud3• 17 years ago

It's monolithic but modular 11.5M LOC for every platform it runs on, not to mention drivers for the devices it supports, and all the modules for other functionality, pluggable file-systems and networking stacks. Now throw in code for backward compatibility going further back at least to 2.4 and you have 11.5M lines.

In reality, the core Linux kernel is just the stuff that compiles to vmlinux. You could read it all in a few weeks and understand it.

skwiddor6• 17 years ago

thats right, bloat

filesystems dont belong in the kernel.

ecq5• 17 years ago

it's a monolithic kernel what did he expect.

Remember the Tanenbaum vs. Torvalds debate?

I wonder what Linus thinks about it now.

btw, here they are together.

http://lwn.net/images/conf/lca2007/lt-ast.jpg

Tuna-Fish5• 17 years ago

I'm sorry, but do you have any idea what you are talking about?

How the kernel is organized has no meaning on how bloated it is. A microkernel can be just as bloated, in it the bloat is just not in the kernel itself, but in the processes providing the services a monolithic kernel would provide. If anything, a microkernel is always more bloated because of the extra code needed to do all the process sync.

Linux being a microkernel would make the situation worse.

(What we need to do is to start spending more time on cutting features.)

ecq5• 17 years ago

you're wrong on several points i don't know where to begin. sorry i don't agree with you.

ecq5• 17 years ago

Compare Linux and QNX (http://en.wikipedia.org/wiki/QNX)

and tell me which one is bloated.

tensor2• 17 years ago

Does bloat mean too many features? In that case I'd say those features are there because someone needs them. They don't all have to be loaded. In a monolithic kernel that has modules, load the ones you need. In a microkernel, load and unload as needed on the go.

Then there is the support issue. In Linux, all the modules must be maintained by the kernel team. This may constitute bloat as the team may have an unnecessary amount of code to maintain. Some modules may only be used by a few people. The issue isn't the features, but rather Linus's philosophy.

From his posts, it seems he like to keep changing kernel APIs as he sees fit. If you have many modules, this creates problems. Without a strict API, you will frequently break modules and thus create a lot of maintenance work. Fixed APIs are both very important and very good for large scale collaboration. Without standards like POSIX or the Windows APIs, we wouldn't be where we are today.

The second aspect to this is security. Say you built a very stable API and let others make modules as they need them. In a monolithic model, to maintain security you must audit every module. Thus for Linux to stay secure, the kernel team would need to either declare many less used modules as "tainting the kernel" or audit them all: a lot of work. In a microkernel model, the kernel team, so long as they designed it right, can let users build modules without audit.

Saying something has too much bloat is ambiguous to the point of being useless. Many people use that term simply as meaning slow, or a lot of code, or even poorly designed code. It's always better to describe things precisely. If it has too many features, say so. If it has bad design, then state that.

skwiddor6• 17 years ago

> If anything, a microkernel is always more bloated because of the extra code needed to do all the process sync.

there are more than two types.

We have fretted recently because the plan9 kernel is struggling to fit on a floppy disk when loaded up as an installer with most things turned on.

The whole of plan9, kernel, userspace, 386 binaries & sources for 6 architectures fit into 200Mb uncompressed (and the whole shooting match compiles in 15 minutes).

Linux is a re-implementation of an OS that was already considered dead by it's maintainers.

"Not only is UNIX dead, it's starting to smell really bad." Rob Pike circa 1991.

see also "Sometimes when you fill a vacuum, it still sucks."

acg• 17 years ago

How the kernel is organized has no meaning on how bloated it is

Software organization can cause bloat if the feature can be implemented in another way. For example, there may be multiple ways to establish the same network connection.

I'm not defending microkernels, but some kernel functions in userspace is not bad. Dismissing microkernel concepts immediately is just as bad as questioning whether Linus was wrong in not selecting that architecture.

morphir• 17 years ago

who of them should have had on the t-shirt *Im_with_stupid ?

zouhair• 17 years ago

Now micro kernel doesn't seem that stupid after all?

snprbob862• 17 years ago

Despite the snarky tone of your comment, you hint at an interesting point:

Unix is NOT the be-all, end-all definitive operating system. Linux is NOT even the be-all, end-all Unix (or Unix-like). There is A LOT of research left to be done in operating systems / kernels and many many years of evolution to come.

It's a shame that it is SO HARD to build the ecosystem required to launch a kernel. New ideas are hard pressed to gain traction.

pyre2• 17 years ago

You realize that OS X uses a micro-kernel, right? And that GNU Hurd is a micro-kernel too?

tomjen2• 17 years ago

Is it really so hard to gain traction? Just write a UNIX micro kernel and add a compatibility layer for Linux device drivers, then you are set.

likpok• 17 years ago

Launching a kernel is easy.

Making people want to use it is hard, as most apps people want are boring to write and/or trapped in the poor design decisions of the past (e.g. POSIX).

CitizenKane• 17 years ago

A micro kernel doesn't necessarily mean that there will be less bloat. It just means that different parts of the functionality of the kernel are split up into different programs.

But those programs have to talk to each other, and this message passing overhead increases with increased functionality (more programs). So it seems likely that there would be a definite performance drop as the functionality of a micro kernel increased.

yason• 17 years ago

How would it reduce the bloat mentioned in the article? Would there be somehow less code to implement the same features in a microkernel architecture?

m0th872• 17 years ago

Mot less code to implement, but it modularizes logic such that it's (theoretically) trivial to disable unneeded features. I'm sure the majority of that Linux kernel logic is never executed; if that code were instead placed in user-level processes, they would have no performance hit.

There are a lot of other side benefits. As stable as Linux is, it still has a great deal of code executing in kernel mode, which is error-prone; microkernels can restart crashed processes on-the-fly without compromising the stability of the system rather easily. Also, some of the newer microkernels have formal proofs of their security properties, which is pretty damn fascinating! (albeit they assume the underlying proof system itself is error-free, which might not be true)

tybris• 17 years ago

Microkernels are not superior from a technical perspective, but from a software engineering perspective.

-Monolithic kernels do not enforce modular implementations, inevitably leading to unnecessary dependencies

-Monolithic kernels need to be developed by a single party, who could not possibly manage such a large code base. The result is poorly maintained code.

-Monolithic kernels need to be recompiled to turn off features (and I have better things to do)

-Microkernels can tolerate more bugs (may be good or bad)

-probably a couple more

known2• 17 years ago

Linus is getting old.

scythe3• 17 years ago

Is it really possible to implement the full functionality of the Linux kernel in such a way that it would be significantly smaller? Obviously, there's bloat from bad code, but would a redesign of the kernel be seriously useful, or just a waste of time? As a (possible) comparison, is NetBSD's kernel any smaller?

It'd be nice to know if Linus had anything to say to that effect - it might lead to a more productive discussion than waxing lyrical about the wonders of microkernels.

davidw• 17 years ago

> Torvalds answered. "Acceptable and avoidable are two different things. It's unacceptable but it's also probably unavoidable."

I think that's more or less where things are right now. You can't have tons of features without paying a price. They do a pretty good job of balancing things, but maybe someone will come along and improve things in the future.

pmjordan• 17 years ago

Rather than bringing out the old monolithic vs microkernel chestnut, let's hear some realistic ideas on how to cut down the bloat.

Does anyone know how much redundant code is in Linux right now? Would finding, isolating, generalising and merging it be a good way to start? I know of examples of this happening, e.g. libata taking over a lot of the old 'ide' drivers; the many attempts at unifying the various WLAN subsystems and drivers.

rythie2• 17 years ago

Whilst libata and the unifying of the WLAN may make the code easier to maintain, it will only save a tiny amount of memory and in the short term, just like any new code, is going to increase the number of bugs.

It sounds like the problem is that everyone is writing new features that probably not many people actually need (since they did fine with out it before - I'm not talking about drivers here) but it's slowing the whole kernel down and people need to go back and optimise that stuff.

robryan• 17 years ago

That's the problem though isn't it, you start off light weight and you continue working, adding more and more new and useful features until you have it all but now have a giant slow bloated system.

ilitirit• 17 years ago

I don't understand why he mentioned this.

I understand software bloat to describe the situation that arises when unnecessary features added to a program increases it's size and/or decreases performance, without providing significant user benefits.

In the article, Linus says "Acceptable and avoidable are two different things. It's unacceptable but it's also probably unavoidable."

If it's unavoidable, then the features that were added aren't really bloat, because it implies that these features are necessary, and that they provide significant benefits. Necessary additions that provide significant benefits could be anything from security features to IO.

IMO, Linus should either state what he thinks should be removed from the kernel, or he shouldn't even bother mentioning it at all.

If on the other hand he's lamenting the fact that there's a cost associated with adding features (necessary or unnecessary), well then I think most people understand that you can't have your cake and eat it, so even from that perspective his point seems redundant.

jsz0• 17 years ago

It's easy to make a tiny super efficient kernel. Just don't support stuff. WLAN? Who needs it? SMP? That'll never catch on. Take it out. Device driver support? Bleh. Let hardware makers post the source code on their websites and users can download and compile the code. No need to include it in the kernel source tree. And who uses anything but x86 anymore? Get all that PPC, ARM, SPARC non-sense out of there too. If we all work together we can make Linux completely useless but hey, it'll be tiny and efficient. We can sit around and congratulate each other for being such elegant programmers. We won't have to worry about those pesky end users anymore either.

dkarl4• 17 years ago

Device drivers and processor architectures might make the source tree big, but they don't make the kernel slower or its memory footprint larger. I believe you can even still compile the kernel without SMP support.

known2• 17 years ago

Who should decide if a piece of code to run in kernel space or user space?

    user/developer/management/customer/congress/judiciary

stinkytaco2• 17 years ago

How about "Linux Lite"? I know, it's probably a silly idea, but is it possible that the kernel team could provide a "core" kernel, that runs on most modern hardware, most popular filesystems, etc. Shoot for some arbitrary support level (80% of systems). Also provide a "large" kernel with everything. Distro maintainers can use either.

Actually, that brings up another point. Who, besides Slackware maybe, uses a stock kernel? Don't most distros customize it anyway? Am I being unfair in saying most Linux users don't really encounter the kernel at all?

Linus says Linux has become 'bloated and huge'

Related Stories