chibea

One main problem that we observed was that big parts of their IAM / auth setup was overloaded / down which led to all kinds of cascading problems. It sounds as if Dynamo was reported to be a root cause, so is IAM dependent on dynamo internally?

Of course, such a large control plane system has all kinds of complex dependency chains. Auth/IAM seems like such a potentially (global) SPOF that you'd like to reduce dependencies to an absolute minimum. On the other hand, it's also the place that needs really good scalability, consistency, etc. so you probably like to use the battle proof DB infrastructure you already have in place. Does that mean you will end up with a complex cyclic dependency that needs complex bootstrapping when it goes down? Or how is that handled?

It's a bit funny that they say "most service operations are succeeding normally now" when, in fact, you cannot yet launch or terminate new EC2 instance, which is basically the defining feature of the cloud...

The max-unused percentage feature is well worth it to 80/20 the prune process and only prune the data which is easiest to prune away (i.e. not try to remove small files big packs but focus on packs which have lots of garbage).

In general, there's an unavoidable trade-off between creating many small packs (harder on metadata throughout the system, inside restic and on the backing store but more efficient to prune) versus creating big packs which are more easy on the metadata but might create big repack cost.

I guess a bit more intelligent repacking could avoid some of that cost by packing stuff together that might be more likely to get pruned together.

Isn't he dismissing culture with his rant? Culture as the currently common way to do and describe things?

He seems to imply that there could be an easier, more straight-forward way to describe things in some more common language. And that while he doesn't give any evidence how the current ways are overly complex.

Of course, there is broken or outdated software, and some things were crap from the start. Of course, there are always concrete things to improve but you won't get anywhere by dismissing all of it and starting anew.

For me, understanding the current state as part of our culture and our humanity and improving gradually on it, has guided me well in the past.

That's implication not equation.

Actually, that the malware contained a root exploit is fixable sooner or later.

The next insight will be, that even if the sandbox had worked, this type of attack still is possible by using the user's trust in the brand of a well-known app to use the permissions granted to it for malicious intents. There's no easy way to avoid that up front automatically.

The downside is that a bunch of processes started from one TTY doesn't get as much CPU as before. It basically shifts the scheduling granularity a level higher from processes to (interactive) sessions. Because that's what the question is: on which level do we want to have a fair scheduling? For a desktop user, processes have little meaning. Sessions, instead, are much more useful because they correspond better to his different tasks, for which he expects that the CPU power is distributed in a fair way.

What's the reasoning for your reproduction of the complete text here? With all possibly valid reasoning in this particular case, there should be high barriers for just copying someone else's text, esp. here given its personal nature.

Often, manual allocation is even slower.

http://lambda-the-ultimate.org/node/2552

An interesting side note here is that the coming Unity will be based upon compiz, instead of relying on mutter. Canonical apparently hired the lead developer of compiz:

http://smspillaz.wordpress.com/2010/10/25/a-bright-new-futur...

"and before long you have a system that demonstrates rare flashes of brilliance but in practice is so unpredictable and opaque you can't use it"

_That_ actually seems to resemble human psychology to some degree ...

Yeah, it seems it does, quite impressive!

I think that's a valid point. For long years mostly the live performance of media was paid for. Then with the manifestation of media as data, the guys who had the machines to record and create copies claimed the stakes and the exclusive rights as a form to create economic value. Time has passed and now we have found out that every data are equal and data can hardly be bounded and the exclusivity of copying has gone away. Perhaps it's time to rethink the claims.

No news, but still worth pointing to...

(e.g. http://blowery.org/2008/04/08/opendns-is-proxying-google/)

I think so, too. If you think about what is making Android devices run so slow, it's almost always some misbehaving background services. Unless there isn't some smarter way controlling the scheduling of foreground applications, there won't be multitasking and fast speed at the same time with a comparable CPU.

"The GNU Hurd is under active development. Because of that, there is no stable version." -- http://www.gnu.org/software/hurd/hurd.html

"[...] so for those hoping for an alternative to minix-386, please ignore me"

Sorry. Won't do you this favor...

About Javascript: "Javascript is a fairly poorly thought-out language. Considering its influences were Scheme and Smalltalk it's rather sad what came out - but it got lucky, it's in a crucial place, it simply fell into the right place in the universe."

Absolutely correct.

I had great expectations when I heard from Wolfram Alpha. The things Mathematica is able to do backed by a giant structured data store seemed mind blowing. As the author said the visualization tools and the (still small) dataset _is_ impressive. I thought: We have all this data, all this visualization options and now we can create aggregate and munch all the facts of the world with an ad-hoc query language. But this simply does not work. There is hardly any functionality to really process data (aside from the predefined ways). And if you come to think about it: One cannot think of a way more complex transformations could be expressed with WA's natural language interface.

For me a better interface would look like this: The basic interface is some kind of full-featured expression language to navigate the data hierarchy (perhaps similar to SQL). You could then build some GUI to interactively navigate/parametrize data/transformations to build more complex expressions. To make the system more easily accessible you could then - and only then - add the natural language recognition system on top of this, which tries to guess some expressions from your input string and gives you some suggestions to start from.

The authors speculate that this is because some people just can't handle the meaninglessness of programming

I think he cited the paper wrong here: It's not the meaninglessness of programming but the meaninglessness of programs. And it is obviously not meant, that a program makes no sense, but that a formalism can stand on its own, without any purpose. Programming, however, is creating programs for a purpose, thus, abstracting facts into a formalism and after running the program interpreting its results is not meaningless at all. I think I can back the observations of the original paper. I occured people which seem to plainly don't get what a program does. They are not talking about what they themselves told the computer to do but they are speculating why the program/computer isn't doing what it ought to do (what its purpose is).

I disagree with Steve's opinion that the level of abstraction is a general problem. I started programming early, and I hit walls all the time. Back from the early times to now I always found things I just couldn't grok. For long years I couldn't understand programming graphical things at all. I watched with awe how my three years older neighbor could program animations on his C64, all I managed to do were Fahrenheit to Celsius converters. Later I was crazy enough to start learning C++ with only knowing BASIC before. It took days to get 'Hello World' running. I kept trying and eventually I found the right book which made me understand OOP. Programming some graphics came very late. And the list still goes on... I can read a book without understand a thing. I come back to the book (or suddenly remember details) some weeks, months, years later and manage to understand what is going on.

The same applies to other areas: Playing piano is something where you hit your personal wall all the time. Sure, there is a natural anatomical caused level of virtuosity which simply can't be overcome. (See Robert Schumann, who tried to hack his weakest finger which ultimately failed) But there are "mental barriers" which can only be overcome (aside from exercising) by keeping cool and sometimes stop trying to come back to it later.

Another example: As a child I always divided the physical world from the living world (the christian heritage?). As such it was for a long time unthinkable that creatures are made from the same stuff as everything else. I recall that accepting this knowledge was really hard...

Regarding the original paper (they've done much research, I can only guess and tell from observation): The humps they observe, couldn't they just be the indication of a clear mental barrier which has to be overcome? Perhaps being taught and some pressure is just not enough to grok these abstract facts about programming? Perhaps, what is needed is just time to (un)consciously chew on the ideas? So, shouldn't be children exposed to 'some programming stuff' as early as possible?

In Germany at least it is left to chance when children become exposed to programming. In most regions there is no teaching at all before a pupil is about 16 years old (11-th grade).

Not talking about immutability at all when talking about CouchDB's (potential?) scaling ability is odd.

Even if immutability in DBs was no innovation from CouchDB itself, CouchDB will have raised the level of awareness for immutable DBs and probably drive further development in this area. IMO immutability is the very basis of creating scalable applications. So, when talking about CouchDB and scalability you should talk about why the current CouchDB implementation is not scalable in spite of immutability.

Disclaimer: I've only read about CouchDB and never actually used it. I'm not convinced of every feature of CouchDB, I don't think, for example, that it is a good approach to dump type-safety in dbs altogether, though, SQL and RDBMS are not the last word in databases and it is valueable to test other, new models of data storage.

Merging: join two (or more) branches of history. The result is a new commit, which has all the merged commits as parents.

A1

|\

A2 B1

A3 B2

| B3

A4 |

| /

A5 (parents: A4,B3)

Cherry-picking: Pick one specific commit from another branch and apply it to the current branch. On your branch the commit gets a new hash (a new identity) since the commit identity is based of a) the parents of a commit and b) the file contents at this commit. There is no information left where the commit came from (aside from author and date).

A1

|\

A2 B1

A3 B2

| B3

A4

|

A5 resulting from cherry-picking B2

So both methods serve a different purpose: merging brings all of the changes of a branch into another branch; cherry-picking just brings in the changes from one commit into a branch. If you want to bring just one particular change into your branch there's no way around cherry-picking.

Another approach is to develop particular things in a topic branch from the start. If you like all of the changes in this branch you can merge it later.

The aversion against cherry-picking and its bigger brother rebasing (which basically cherry-picks all the commits of another branch) is based on the basic principle, that each commit represents a somewhat tested state of a program in time. All VCS want to ensure you can go back to these tested states later. Later, if a bug occurs you want to be able to find the commit where the bug was introduced, you can use git's bisect functionality to find this specific commit. There can be two types of bugs: original bugs and integration bugs. In the chery-picking/rebase case you have only one commit for a feature so you cannot easily decide if it is was an original bug or instead introduced by integrating a specific feature. Using merging, you end up having one commit for the feature and one for the merging (integration) and this can help in particular work-flows.

You might want to see discussions with and by Linus Torvalds on the linux kernel mailing list about this topic...

Right, it would be more interesting to talk about memory consumption and if that might add to power consumption.

Right, google does not pay the bill, but that's not the point of android. It's meant as a platform for developers which then don't have to worry about CPU-specifics. And as important, it is meant as a platform for mobile device vendors. Though, google had to make sure that dalvik has a reasonable performance compared to native apps. Even if android Java apps are 2-10x slower than corresponding native applications, that's ok, because further savings will come with the next level of power-saving CPUs for mobile devices which will be able to run current android apps unchanged.

Are there any compiler features which change the ratio power consumption/clock cycles? If yes, that's what he should have talked about and give some evidence.

If no, let's assume power consumption is a linear function of clock cycles and focus on cycles.

He claims JIT would cause 'periodically run background optimizer tasks'. He does not actually state if he is talking about single or multi-processor systems. Let's assume first, he is talking about a single-processor system. Then, the background tasks are actually run on the same CPU and thus are counted in the benchmarks. If he, instead, is talking about a multi-processor (-core) system, he claims that even in a single threaded benchmark, there might be hidden cost (hidden, since runtime is improved by 'stealing' cycles from another CPU) inflected by these background tasks. He is probably right here, since besides JIT, there is e.g. garbage collection which in fact might be running concurrently on another thread. If you only measure runtime of such a benchmark it won't be an accurate measure of the power consumption. But: If you measure CPU-time/cycles of the benchmark all the threads are taken into account with all the 'hidden' costs. He provides no evidence that the benchmarks failed to do this.

What he could have meant is that some benchmarks use 'warm-up' to have all the hot spots JITed before measuring. He's right here: for short running tasks JIT has to be taken into account. For long running (massive data processing) tasks it is at least not obvious that JITing is slower or faster than ahead-of-time compiling. There are some opportunities for a JIT-compiler to actually use less cycles than AOT-compiled code. E.g. a virtual method call in C++ has to use the virtual function table always, this indirection might introduce additional cycles due to bad locality. A JIT compiler knowing that a particular interface call at a call-site in fact is monomorphic, can optimistically skip one level of indirection and thus save cycles.

The vendors of JIT compiler put much effort in the balance between JIT overhead and possible performance gain. One can assume that they know how to create meaningful benchmarks. If in doubt, provide counter-examples.

BTW: The correct unit of comparison would have been work/joule and not work/watt.