“DBus is seriously screwed up”

http://thread.gmane.org/gmane.linux.kernel/1930358/focus=1939166

by deng • 11 years ago

281 191 11 years ago

thread.gmane.org

But, let's stop bitching and come up with alternatives, solutions, or closure. I'm tired of hearing about KDBUS (all love to gregkh) and DBUS.

rdtsc• 11 years ago

Good link.

As a side note, I see sorokin's posts about DCOM, RPC and multi-threading:

Probably some unique thread-id can be propagated through the calls. And if incoming call has the same thread-id as one of our outgoing call it is handled inside this outgoing call as in nested message loop, otherwise a new thread from thread pool is used. This will create an illusion that two processes share the same set of threads and it works well with mutexes.

I think he is slowly driving towards an actor like message passing thing like Erlang, Akka or Orleans without even realizing.

Which reminds me of Virding's law:

http://rvirding.blogspot.com/2008/01/virdings-first-rule-of-...

---

Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.

---

userbinator2• 11 years ago

alternatives, solutions, or closure

The link you posted has what I think is the right one:

The problem is that probably this code is not needed at all.

To me, DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives.

bandrami3• 11 years ago

there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives

This. So many things that freedesktop does I just kind of look at and think "this just isn't remotely a problem I have, so I can't judge whether it 'solves' it or not."

bitwize6• 11 years ago

Then you need to read and reread and reread Havoc Pennington's posts on why he wrote d-bus in the first place:

https://news.ycombinator.com/item?id=8648995

https://news.ycombinator.com/item?id=8649459

D-Bus provides a service discovery system that lets you not only send messages to named endpoints, but also monitor whether there's something on the other end of the endpoint, and if necessary, request that it be started before you send the message in a non-racy fashion.

It also provides a common, typed, structured, discoverable, auditable serialization layer which is necessary for doing API-style IPC, and a benefit besides since when you "just use sockets" you have to do all the serialization and deserialization yourself, leading to possibly buggy code.

Oh, and it also does multicast, which no other commonly used Unix IPC can do.

People who say "I don't see the problem that d-bus solves" aren't looking hard enough and/or are ignorant of the issues of how software gets developed in a modern environment. D-bus solves many problems with IPC under Linux, and makes developers' lives a whole lot easier when they have to connect with other applications or services.

byuu3• 11 years ago

DBUS seems like a solution looking for a problem: The majority of the time there is no real reason to have such a complex layer of abstraction for IPC, when the system already provides much simpler alternatives.

Indeed. I just use shm_open()+mmap(MAP_SHARED) and go about sharing my data between processes, and have had no problems whatsoever. Then throw in sem_open() so only one process is locking the data at a time. Wrap all that in a 2KB header and it's trivial to use in any project.

It would seem that modern computing is all about building abstractions on top of abstractions, to solve 'problems' nobody ever had. Nobody ever seems to consider the costs of such complexity (in terms of maintenance, understanding the system as a whole, attack vectors, etc.)

By all means, if you need a super complex system with tons of features like D-bus, great! But I am pretty confident that Mousepad (Xfce text editor) doesn't need D-bus just to open a text file in a new tab of an existing window.

digi_owl6• 11 years ago

DBUS is a "weird" one.

It came out of the initial freedesktop project to improve interaction between KDE and Gnome.

KDE already had system called DCOP for internal use (allowed for instance Konqueror to be web browser, file manager and multi-pane FTP/SFTP client all at once).

But as with much of KDE it was created using C++. And Gnome is made using C.

So DBUS is something like a reimplementation of DCOP in C.

And the rest is history...

bitwize6• 11 years ago

People have come up with alternatives. They are:

* AF_BUS, a new socket address family that provides a "generic" bus

* adding multicast and order guarantees to AF_UNIX sockets

None of these have the political traction that kdbus has. Kdbus, systemd, etc. get mainstreamed because they have top developers gunning for them being mainstreamed. The people on the sidelines going "but muh unix" are also-ran haters, nothing more. So let's just throw the switch and fry this sucker.

joelthelion• 11 years ago

He's not saying DBus as a concept is bad, just that the implementation sucks. If he's right, that's something that can be fixed relatively easily (compared to a major architectural change).

TD-Linux2• 11 years ago

I agree, the OP title is a bit misleading.

I use dbus every day on my laptop and it works totally fine - I don't notice its existence. I'm all for improvements, but I'm quite patient and quite happy to wait this one out.

reidrac3• 11 years ago

I made a silly desktop app some time ago and it used DBUS to get notifications from NetworkMonitor when the system went online/offline. Nothing too fancy, very few lines of code.

When I was implementing that, I managed to get several segfaults from my Python code. All together seemed a little bit fragile to me :(

That was 5 years ago, things are probably better now (to be fair, I don't know what was causing the crashes; NM was 0.8 back then), but when I read Linus comments I can help to think he's probably right.

This is an anecdote and all, but my point is that until I had to use it... DBUS was pretty good and was working fine :)

dingaling• 11 years ago

I use dbus every day on my laptop and it works totally fine

My laptop appears to run fine despite dbus, if the hundreds of daily 'Failed to connect to socket' messages are any indication.

anc84• 11 years ago

Maybe your laptop would be much faster if dbus was faster.

rasz_pl• 11 years ago

no, both concept and implementation is bad. This thing shovels and computes a LOT of data per call, you cant optimize that out because its what authors of kdbus intended. Its typical 'look at all that CPU we have now, lets use it' mentality that keeps Wirth's law true.

A_COMPUTER• 11 years ago

He does seem to be saying that kdbus is pointless since those fixes would be in userspace where it's spending most of its time. Or am I misunderstanding?

masklinn• 11 years ago

The point of kdbus is to do dbus in kernel space. The one spending lots of time in userspace overhead (not doing actually useful work) is regular dbus.

Linus is saying that kdbus is pointless because its performance gains don't come from being in-kernel, they come from the code not being a complete shit-show, and he believes the same performance should be achievable by fixing regular userspace dbus.

castell4• 11 years ago

Since then I'm convinced that people who are inventing RPC solve non-existing problems.

Many RPC solutions ultimately failed, because they were slow and overdesigned/complex. (e.g. CORBA, Network OLE/DCOM, Java RMI, XML-RPC/SOAP and other XML-based protocols, etc.)

Whereas e.g. REST (if you call it even RPC) is just very simple and enough for most purposes.

It's more like the concept of Component Object Model failed. For example OLE (the base technology behind DCOM) doesn't fly beside the legacy Office usage. It goes without saying that binary based implementations of such Component Object Model are faster than XML based ones - a fade of the last ten years.

The component format (if you call it that way) that just works is HTML5.

Edit: okay, it seems it's a controversial topic (My comment used examples which were DCOM and OLE 1+2 (the original MS Office thing "Compound document") - http://en.wikipedia.org/wiki/Object_Linking_and_Embedding and http://en.wikipedia.org/wiki/Compound_document , not COM directly)

vidoc3• 11 years ago

The problem with REST is that it's great for what it was intended to do initially, but there are too many engineers that don't take their medication that actually use it to do RPC :P

lclarkmichalek2• 11 years ago

Naive REST (HTTP, JSON) isn't exactly known for its performance

vidoc3• 11 years ago

Agreed, that said, I'm kind of wondering what non-naive (smart?) REST is, no sarcasms :P

thezilch• 11 years ago

Seems to be doing better than the speeds sited by the KDBUS thread -- https://www.techempower.com/benchmarks/#section=data-r10&tes...

anton_gogolev• 11 years ago

Apples and protocols.

REST is an architectural approach, and it can very well be using the same overdesigned XML envelopes, binary-encoded payload or anything else, really.

roel_v2• 11 years ago

You do realize that pretty much whole of Windows is, underneath, based on COM, including most of the .Net-exposed system API's?

I'll agree that COM and DCOM are... hairy, if you're implementing COM objects 'by hand'. But there's a reason for all of it, and I have yet to see an easier and more flexible way to write cross-language components. I could write a COM object in C++, call it from VBScript in a very simple command line program, as well as use it directly in a Web application in (here it comes!) 1999! That's 15 years ago! And while COM is universally reviled now, and none of the cool kids will want to be seen within 100 feet of it (actually, the 'cool kids' don't even know what COM is any more, it's so 2005 to hate on COM...), for those who spend 6 months on understanding it, it worked very well, and has been supported for close to 20 years now (on Windows, that is).

72deluxe2• 11 years ago

COM is indeed an interesting beast and very useful. Being able to integrate other programs into yours, relying on COM to do so is very helpful. I don't know of a way on Linux or OSX to embed a word processor to work on documents but never show the user (but this might be due to my lack of knowledge about those systems - I would welcome being enlightened). I once wrote something that processed a plethora of Word documents using Word via COM to extract data from them and shove it into a database.

I can't fully remember how I did it but I recall type libraries, including generated headers etc. but I remember being impressed by it.

And as you state, all calls within Windows rely on COM. Stop the RPC service and observe as your system becomes unusable.

mnkypete• 11 years ago

It is still working very well indeed. I was recently trying to add photos to iCloud programmatically on Windows. Guess what, there is a COM interface for that.

https://gist.github.com/tobiasviehweger/7a302b7179efb99082d8

ntoshev• 11 years ago

COM sort of works, on Windows, when when one or preferably both sides of the connecting components are made by Microsoft. You never see non-Microsoft code talking to non-Microsoft code from a different vendor via COM. This is not a successful component interface standard.

bandrami3• 11 years ago

It's more like the concept of Component Object Model failed. For example OLE (the base technology behind DCOM) doesn't fly beside the legacy Office usage.

True, but the idea of the self-registering COM object eventually seemed to limp across the finish line.

Anyways. "Two key kdbus developers" (ahem) have said that the entire kernel signal mechanism should be deprecated because it's "too brittle and complex". When I read that, a big light bulb turned on in my head about what the actual problem is here.

XorNot2• 11 years ago

Except they're not wrong. We have a potentially infinite set of signals and data we'd like to communicate to programs and a very limited set of signals to do it with. I mean apache uses sigwinch as an exit code!

elwin• 11 years ago

Signals are too brittle and complex. That's why you have to read three whole manpages to figure out what happens if a process gets the same signal twice in rapid succession.

They are also un-Unixlike: they are used to communicate three or four different kinds of information, and they do most of them badly.

1wd• 11 years ago

Isn't Mozilla's XPCOM (the basis of Firefox and all Mozilla applications?) also based on COM?

https://en.wikipedia.org/wiki/XPCOM

lmedinas2• 11 years ago

Another great Linus quote:

"The people who talk about how kdbus improves performance are just full of sh*t."

http://thread.gmane.org/gmane.linux.kernel/1930358/focus=193...

pmr_• 11 years ago

You are not quoting the part where he shows his benchmark results, analysis of the problem, and suggested solution. I don't necessarily agree with Linus' tone or choice of words, but what you are doing is far worse. I'm not even sure if you are trying to be sarcastic, but that does not even matter: You are not contributing anything either way.

lmedinas2• 11 years ago

I have made an observation from Linus comments which is sad specially to people who worked on KDbus all this time. If you think your comment helped anything maybe you should consider making them when you see comments like the next one:

quote: """ amelius 3 hours ago

Should be rewritten in Erlang or Go, if you'd ask me. """

otherwise you are full of sh*t. :)

mwfunk• 11 years ago

As entertaining as it might be to people who are entertained by such things ("not me" would be an extreme understatement), it's pretty much exhibit A for Linus' inability to manage people in addition to code. It's embarrassing, honestly.

anon42• 11 years ago

Well, as Eric put it

At this point the strongest possible language and the strongest possible push back are being used because everything else is routinely swept under the rug.

hp7• 11 years ago

Linus is mixing together two different performance issues.

http://article.gmane.org/gmane.linux.kernel/1939651

kdbus is solving an "architecture" issue that would affect any binding, while his profile is essentially of the gdbus binding.

vidoc3• 11 years ago

It all started with a microbenchmark:

http://thread.gmane.org/gmane.linux.kernel/1930358/focus=193...

DBUS implementation aside (which seems to leave a lot of room for optimization), my initial reaction was that the gain was rather minimal compared to the claims of the systemd folks.

deng3• 11 years ago

That was my first reaction as well. But to be fair, the kdbus people always said the real speedup will be seen for large packets.

However, the real issue is how slow overall DBus seems to be according to those numbers. As Linus says earlier in the thread: "No way should it take 4+ seconds to send a 1000b message to back and forth 20k times." This is indeed rather shocking.

lmilcin3• 11 years ago

I would expect any half decent HTTP server to be able to handle more messages per second than that. If that is going to be the future fabric underlying all application ecosystem on Linux I would be as wary of it as Linus is.

I will put it another way: if it can't be more performant than a simple HTTP client/server it doesn't deserve to be included into mainline kernel especially when the costs are so high.

fulafel2• 11 years ago

So what's the use case for sending large packets of data over dbus where the performance gains warrant integrating the dbus server to the kernel? Are they planning to migrate the X server to dbus?

anon42• 11 years ago

In that case, they should implement just the functionality they need in kernel, not a full DBUS implementation.

riking• 11 years ago

So, basically, systemd-bus would solve the problems even without kdbus, if it avoided all the extraneous memory allocations.

audidude2• 11 years ago

I don't think Linus is up to date on libgio-2.0 (and the GDBus implementation).

It does in fact use a SLAB, based on early kernel versions with per-thread allocation caching.

As for avoiding utf8 validation, that would require being able to trust the sender/receiver. That, I believe, is a major bullet point of both memfd (sealing) and kdbus.

hp7• 11 years ago

kdbus really addresses a different performance issue than tuning mallocs and locks would. By tuning mallocs/locks/validation/parsing, you can reduce the overhead of each dbus send/receive, but with kdbus you remove half of the send/receives. So however fast your sends/receives are, kdbus will still make things faster by not doing as many of them.

pjc50• 11 years ago

Some form of notification publish/subscribe functionality is necessary in modern Linux. But dbus isn't very UNIX-y. It's also strangely opaque. With pipelines, you can see how the plumbing works. With dbus, programs can interact in nonobvious ways. Even worse is polkit, a poor replication of Windows Group Policy.

The most unixy solution is buried at the bottom of this thread, from Plan 9: https://news.ycombinator.com/item?id=9450988 ; but if that's not popular, then I think what people actually want for desktop purposes is an equivalent of the Windows PostMessage system.

XorNot2• 11 years ago

Plan 9 in no way implies performance. Just because you can make it look like a filesystem, doesn't mean it actually is going to achieve anything special performance wise.

jff• 11 years ago

And in fact the performance of Plan 9 tended to be extremely poor when compared apples-to-apples with Linux.

It's just that everything was so simple it felt fast, even if your filesystem throughput was crap.

lmilcin3• 11 years ago

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that.

Heck, I have worked on an algorithmic trading platform that in the limit of 5us receives market data, dedups it (multiple multicast streams for redundancy), uncompresses it (fricking zlib), parses it, analyzes it, sends to multiple algorithms which decide if current market situation matches certain rules, decides and fills market order, the order gets inspected by independent mechanism to stop the algorithm if it malfunctions, and only then it gets sent to market, over TCP, which is another form of IPC.

All in the span of fricking 5us which is 40 times less than the benchmark suggests for this simple task. Granted, the algorithmic trading world goes to great lengths to avoid overhead including kernel overhead any kind of task switiching, branch prediction fails, etc. But still, come on, guys...

the_mitsuhiko4• 11 years ago

It blows my mind how such a simple operation as passing messages from process to process can baloon to waste the measured half a million CPU cycles. People manage to have full-blown HTTP servers service a request with less than that.

You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

huhtenberg• 11 years ago

Nothing obvious about it.

If you look at the Linus's trace, it's all heap and mutex operations. That's just sloppy internal design full of concurrency bottlenecks and lots of in-memory cloning. You certainly don't mean to imply that both are the only way to implement a "security policy driven message bus" software, do you?

coldtea4• 11 years ago

You're comparing a lightweight thing with a security policy driven message bus. Obviously the latter is going to be more expensive.

The real problem is he doesn't give any data for volume/bandwidth processed in his case.

I don't see why a "security policy driven message bus" would be more expensive than what he describes (unzipping, business rules encorcement, analyzing, etc).

latitude• 11 years ago

But still, come on, guys...

libnetsnmp used to make close to a million calloc() calls when responding to a query for a number of CPU sockets on the machine. That was kind of amazing in its sheer brazenness.

thuffy6• 11 years ago

Moores Law works to make it eventually more than fast enough.

Moores Law does not help it get more secure.

How about we work on things that mater in this post-Snowden world, like increasing security -- and let Moore do the performance improvements! Seems a more reasonable and efficient approach, no? Adding more attackable surface area to the Kernel is not a good idea at all. It is an NSA wet dream.

vezzy-fnord5• 11 years ago

Moore's law makes it faster, but Wirth's law compensates for that dearly. Let's also not forget that pesky Amdahl making our bottlenecks stay bottlenecks, as some things are just fundamentally wrong for performance.

alextgordon• 11 years ago

Security is usually correlated with efficiency, not inefficiency. Think djbware. If something is inefficient, it means nobody has bothered thinking through the design, and well...

acqq• 11 years ago

The Linus' explanation is here:

http://thread.gmane.org/gmane.linux.kernel/1930358/focus=193...

"Just to make sure, I did a system-wide profile (so that you can actually see the overhead of context switching better), and that didn't change the picture."

"The real problems seem to be in dbus memory management (suggestion: keep a small per-thread cache of those message allocations) and to a smaller degree in the crazy utf8 validation (why the f*ck does it do that anyway?), with some locking problems thrown in for good measure. "

userbinator2• 11 years ago

I can make two observations and hypotheses from the profiling results, which agree with Linus' conclusion of "bad user-level code":

- Memory allocation/deallocation are taking the most time.

- All the percentages for each function are very small.

The former is a characteristic of code which heavily abuses dynamic allocation. It's surprising to see how many programmers are not aware of the overhead it adds and would malloc()/free() frivolously when something simpler would suffice. This is also often accompanied by copious amounts of unnecessarily copying data around. I've worked with small embedded systems where every use of dynamic allocation would need to be justified thoroughly in code reviews; perhaps these developers would benefit from being put through the same process.

The latter is a phenomenon which arises from "excessive modularity": the functionality of the system has been split into so many little pieces that the time each function contributes to the overall total is tiny. Instead of seeing an obvious "80% of the time is being spent here" that could easily be targeted for optimisation, that 80% is scattered amongst several dozen functions each taking 1-2% each. The bottleneck isn't concentrated in one area --- the whole system is uniformly inefficient. It's extremely difficult to optimise a system like this because nothing in particular stands out as being optimisable. I've had to optimise some large Java applications that were like this, and the solution was basically to remove most of the code and rewrite it to get rid of many chains of indirection.

avar• 11 years ago

The way you generally deal with the second problem is to have a profiler that can give you the inclusive time spent in a subroutine, not just the exclusive time. The inclusive time is the time spent in that sub and in subs it calls, the exclusive time is time spent only in that sub.

When you have inclusive time it usually becomes easy to find that function that's spending 80% of the time in either itself or in the things it calls.

FeepingCreature2• 11 years ago

Somebody should modify perf to log "time spent in call", "time spent in stackframe setup", "time spent in mov", "time spent in branches", etc.

DannyBee• 11 years ago

So, you can get some of these kinds of cycle counts by changing the event that gets used for record. https://perf.wiki.kernel.org/index.php/Tutorial#Events

So you could use record -e branch_instructions, for example.

Past that, things like "time spent in stackframe setup" is not sane to measure on an optimized binary, because the instructions hat set up the stack frame may be all over the place (due to shrink wrapping, etc).

This means the overhead of counting them would probably require instrumentation and be quite high (and perf is sampling based, not instrumentation based)

andor• 11 years ago

The percentages are small because

a) This is the output of perf top, a system-wide trace of the last few seconds

b) The application uses lots of library functions

c) Due to lots of locking in the application, the cpu doesn't spend much time in the app itself

To find out which high-level functions (that in turn call library functions) take the most time, use perf record to get call stacks.

yoklov• 11 years ago

Frequently the second problem is exacerbated by cache misses. Every operation becomes (much!) slower, but nothing stands out.

Not a popular opinion here, but excessive modularity like that is practically the definition of bad, hard to follow code though IMO. Ignoring performance issues, everything happens somewhere else, and large changes now take several times longer, since the changes are not local to a function. The code is several times longer than it otherwise would be due to all the function declarations...

Its a nightmare. It tends to be the kind of code you feel productive while writing ("I'm cleaning up this 200 line function"), but is really just making the codebase worse (is there a general term for this kind of false productivity? It's a common problem I see).

_pmf_• 11 years ago

Its a nightmare. It tends to be the kind of code you feel productive while writing ("I'm cleaning up this 200 line function"), but is really just making the codebase worse (is there a general term for this kind of false productivity? It's a common problem I see).

This issue is pervasive when Desktop/Web developers try to improve embedded software. I've achieved a thousandfold increase in performance by converting an embedded data logger from using printf to using a dedicated formatting function (most of the time was spent on parsing the format string and performing allocations).

corford• 11 years ago

For the second problem, sounds like the dbus devs should follow sqlite's lead: http://permalink.gmane.org/gmane.comp.db.sqlite.general/9054...

serve_yay• 11 years ago

Indeed, decomposition is easier than extracting modules. This is often an argument in favor of modularity! :) As others have mentioned, the second issue can be addressed with tooling.

But, is excessive modularity really the culprit? Are you saying you read the code and found that to be the case, or it's just a hunch? Reading through the comments here, it seems that whatever one's personal hobby-horse happens to be is what will get the blame.

GutenYe• 11 years ago

What's potato, as said in the mailing list?

Sharlin• 11 years ago

Internet slang for a device, often a camera, of poor quality or outdated technology.

http://www.urbandictionary.com/define.php?term=potato&defid=...

kzrdude2• 11 years ago

jargon for slow computer or old smartphone.

amelius• 11 years ago

Should be rewritten in Erlang or Go, if you'd ask me.

JosephRedfern• 11 years ago

Are you missing '/s'?

kzrdude2• 11 years ago

But this doesn't benchmark systemd's userland dbus implementation. That one is much better, right?

codys• 11 years ago

I hope so, but no one except systemd is using it yet. Last time I checked the sd-bus.h header isn't installed with systemd yet due to the systemd folks concern that that API for sd-bus wasn't finalized yet.

It would be very interesting to benchmark it against the glib implementation (which most people, even if they aren't using glib directly, are using these days).

hp7• 11 years ago

The Linus profile is mostly a red herring here, because it is 1) a bad benchmark with a bunch of blocking round trips and 2) mostly profiling the gdbus bindings which are just one binding.

For a more in-depth performance discussion of dbus, check out http://lists.freedesktop.org/archives/dbus/2012-March/015024...

Above noted on linux-kernel here: http://thread.gmane.org/gmane.linux.kernel/1930358/focus=193...

zurn3• 11 years ago

The PDF link is sadly dead... (The ml message references heavily to the PDF)

outworlder2• 11 years ago

This whole pulling crap out of the Linux kernel mailing list is doing the industry a disservice.

Quotes are taken out of context, used by people who lack the required technical understanding, which then reach the ears of managers, who will promptly act on overblown concerns. All of it somehow backed by Linus' word, even when he doesn't actually mean it.

I've started a stopwatch to see how long it will take for some of my former coworkers to contact me with concerns about DBus. We used it pretty heavily.

As for DBus itself: guys, it is useful. It works. You can count on it being there and working in most distributions, you can talk to most things nowadays. Its easy to use (for most use cases), language bindings are everywhere.

So, turns out is slow. It's so cool that we have identified that. Let's fix it.

Even if the slowness doesn't seem to bother many of its use cases.

m0th87• 11 years ago

What's being taken out of context here? It seems the conversation is about dbus being slow, and you agree that it is?

tcfunk• 11 years ago

Probably referring to the link title, which could imply quite a bit more than it simply being slow.

forgottenpass2• 11 years ago

This whole pulling crap out of the Linux kernel mailing list is doing the industry a disservice.

The worst part is that I sorta understand why some low-rent blogo-journolist pulls that nonsense. Hacker News should be better than this.

A juicy pullquote can be an attractive headline and cheap and easy to write article. Start with a basic technical introduction paraphrasing wikipedia but doesn't get the reader anywhere near ready to understand the LKML post. Then pad out the rest with some hemming and hawing about: Has Linus gone TOO FAR!?

On HN, these kind of submissions are just as trashy, but because they point directly to LKML we can be fooled into thinking to be something loftier.

It should be called "kdbus patch review: dbus performance overhead may be in userspace library, not context switching." But nobody would submit that because it's not catchy. I'd still click that link, but the title would be boring and technical, just like the post it links to is. I actually flagged the other front page LKML post "Big-Endian is effectively dead."

hp7• 11 years ago

I would call it "dbus performance overhead is userspace library AND context switching" See: http://article.gmane.org/gmane.linux.kernel/1939651

Or really, it isn't "context switching"; it's that there's the bus daemon in the middle which multiplies the userspace library overhead by 2, since messages are read and written twice as many times if you have the daemon in the middle.

Speeding up read/parse and marshal/write is completely separable from doing fewer read/parse marshal/write operations. Two different performance issues that aren't related as far as I can tell.

phjesusthatguy32• 11 years ago

"On HN, these kind of submissions are just as trashy, but because they point directly to LKML we can be fooled into thinking to be something loftier."

No, it's because of the cognitive dissonance in play on HN. It's a fucking news-re-blog, for crying out loud, but because of the stated goals[0] of the site there's some idea that we're rising above the trash and nonsense of sites like Slashdot.

Don't get me wrong; articles get posted here before they get reposted on Slashdot or The Register etc., but that's all there is to HN. It's not a better place for discussion than either of those two sites.

[0]https://news.ycombinator.com/newswelcome.html

stevejones• 11 years ago

We've always known it's slow. It has a ton of other problems as well, not least of which is pointless duplication. But its speed has always been terrible.

drzaiusapelord• 11 years ago

Quotes are taken out of context, used by people who lack the required technical understanding

What? This is literally the maintainer of the Linux kernel on his mailing list voicing his opinion on something kernel related. Funny when he bashes things HN hates like Nvidia drivers, suddenly he's a truth teller and his words repeated on the mountains. When he's bashing something you like suddenly you're "OMG GUISE IGNORE LINUS HE DOESNT KNOW WHAT HES TALKING ABOUT!!"

Come on.

cpwright• 11 years ago

The thing is Linus does know what he is talking about; and can separate the performance concerns, from functionality, etc. I think the real point of the message is to change dbus not the kernel to address the performance issues.

But if someone without the proper technical understanding looks at the just the pull quote, they might think it is a lost cause or not worth using.

ChuckMcM2• 11 years ago

I am wondering if there are "Linux Insurgents", which are groups of people with a particular agenda that find small bits of data which can be made to serve their cause and expose them. Or maybe they just don't like DBus. Or maybe they just want to feel like they are having a larger debate.

If it is any consolation, while I worked at Sun in the kernel group and later, this sort of 'cherry pick a message to start a debate' technique was also present. Sometimes it is just someone who believes something is broken and they use the message to bolster their argument.

nitrogen2• 11 years ago

I also wonder if the reverse happens, with people sabotaging the system by breaking things that work, making easy things harder, and replacing fast things with slower things.

pcwalton• 11 years ago

I am wondering if there are "Linux Insurgents", which are groups of people with a particular agenda that find small bits of data which can be made to serve their cause and expose them. Or maybe they just don't like DBus. Or maybe they just want to feel like they are having a larger debate.

D-Bus is related to systemd (in that systemd relies on it), so this issue seems to have garnered a fair bit of spillover from the larger systemd debate.

felixgallo2• 11 years ago

I don't lack the required technical understanding. The headline is correct; Linus identified several embarrassing and obviously stupid problems with kdbus, including security concerns, in that thread.

npsimons• 11 years ago

So totally this; it's what I've said time and again. Is Linus abrasive? Yes, he can be. He usually has reason. But people seem to rarely read the whole thread, and even if they did, they are usually outsiders who lack both the technical chops to make valid criticisms, and they aren't familiar with Linus' management style. I for one count myself as one of the former as I've been outside of kernel land for quite some time. I'll say this though: the N900, with 256MB of RAM and a single core 600MHz ARM chip from 2009 had DBus, and it didn't seem to suffer too much from it. Hell, it's kind of impressive to be able to make phone calls from the command line, say over SSH:

https://wiki.maemo.org/Phone_control#Make_a_phone_call

Can improvements be made? Probably. Should they be? Why the hell not? Should clickbait headlines stop distracting us from real work? I only wish they would.

vezzy-fnord5• 11 years ago

As much as I loved the N900 for its system transparency and malleability (I still use it), it was hardly a paragon of performance. I don't know how much D-Bus played into it, though.

digi_owl6• 11 years ago

While i enjoyed the N800 for the time, the waves that Maemo has since created across the Linux ecosystem has me wondering if the project should never have happened.

castell4• 11 years ago

Strigi desktop search and NEPOMUK semantic desktop extensively relied on D-Bus. The D-Bus background daemon often hung and NEPOMUK itself was known to be slow and wasteful on hardware resources. Maybe it wasn't D-Bus fault at all, but the years long problems lead to a bad impression. http://en.wikipedia.org/wiki/Strigi , http://en.wikipedia.org/wiki/NEPOMUK_(framework)

Comparing a PPC (WinCE), iPhone1 and N900 with the similar hardware specs - there was no clear winner.

lmilcin3• 11 years ago

I understand the whole point of putting dbus into kernel was to fix performance. The objections are mainly because performance haven't really materialized but the costs are very much present.

kasabali2• 11 years ago

I still fail to see why we need DBUS in kernel, so please correct me and clear my lack of understanding. It is claimed userspace dbus performance is bad, but people want to use it in a much larger scale, so what I deduce from that is there is nobody using dbus on that scale right now, since its performance sucks.

Reason for getting kdbus into kernel, and not a generic ipc but specifically kdbus, is because user space that depends on dbus will continue to work with kdbus.

What I still don't understand is, what is the point of keeping compatibility of currently non-existing userspace, if software that depend on dbus right now doesn't need performance enhancements and can just keep using regular dbus, and software that need high performance bus doesn't even use dbus since it's slow? Under these circumstances, why they insist on making kdbus strictly a dbus implementation but not a more generic ipc mechanism?

thuffy6• 11 years ago

Let's talk about the elephant in the room.

This is quite obviously the NSA adding massive additional attackable surface area to the Linux kernel.

They did it to SSL/TLS, they did it to many others, they are now doing it in earnest to Linux.

First systemd, now kdbus. Keep this up and OpenBSD, or something else, is going to kill the Swiss cheese that Linux is becoming. I love Linux, so it is sad to see this happening.

antocv2• 11 years ago

systemd, kdbus and pulseaudio - wow r00t lol. kthxbai /nsa.

Seriously, its already a huge mess to administate a GNU/Linux with systemd and dbus and polkit already as it is. Its the only parts of my system where I feel I cant really inspect its inner workings. Its a black box.

thuffy6• 11 years ago

I absolutely agree it is a mess for other reasons as well; but let us attack it on all angles, and I think security is the most important one these days.

xorcist• 11 years ago

Now is the time for Linus to disappear for a few days, and re-emerge with a sane IPC design that both binder and dbus can build on.

Bootstrapped in itself, of course :) (I have no idea what that would even mean in this context. But hey, it's all wishful thinking, right?)

phjesusthatguy32• 11 years ago

Can someone point out where the quote comes from? I'm not a DBus developer but I'd like to know why the subject is what it is, and not something like "Issues with capability bits and meta-data in kdbus" or "Kdbus needs meaningful review".

hp7• 11 years ago

It's Linus being hyperbolic. Read follow-ups in the linux-kernel thread to see more specifically what these benchmarks mean.

supertruth• 11 years ago

Linus is just being Linus. He's brusque, non-chalant. He's inflammatory to get people to listen. His thesis here is that bad DBus performance isn't due to context-switching overhead or buffer copies (which can be solved by moving the daemon into the kernel), but instead it's due to malloc-intensive / utf8-parsing-intensive marshalling.

Secondly, he's saying that if performance is being used as an argument for kdbus, then that's an invalid argument.

He's totally right by the way. In this pure message-passing benchmark, where the message-passing overhead is the majority of the work, the slowness is not in kernel-scheduling/system-call/kernel-buffer-copies. People confused a potential impossible-to-overcome bottleneck as the most relevant bottleneck.

But that doesn't mean there isn't a reason for kdbus. Kdbus allows for much better authentication than UNIX sockets do (you can authenticate with pid, pgid, uid, gid, kdbus token, etc.). Also it allows for message-passing security policies to live in the kernel which is crucial for security applications. The tangential performance benefits are nice too, even though the bottleneck wasn't in the kernel to begin with.

voidz• 11 years ago

Responses like yours are why I enjoy to visit Hacker News.

Sorry for going meta - I'm just a bit disappointed because your tone used to be the normal thing on HN, even when there were just as many disagreements. People were able to keep it friendly. Today not as much; I find HN has become too mainstream and "reddity", and the tone in general too emotional and aggressive, about who or what is right.

This wasn't unexpected, but still, it is good to see not everyone has moved into that direction.

“DBus is seriously screwed up”

Related Stories