The Imminent Decentralized Computing Revolution

http://blogs.wsj.com/accelerators/2014/10/10/weekend-read-the-imminent-decentralized-computing-revolution/

by waterlesscloud • 12 years ago

151 79 12 years ago

blogs.wsj.com

jmgrosen• 12 years ago

So the internet is going back to its roots, armed with some new technology? That would make me very happy from a privacy point of view, but I'm dubious of whether it's actually going to happen...

th0ma5• 12 years ago

Anyone tracking the time periods of of the ebb and flow of centralized and decentralized? I guess a lot of things are decentralized and a lot of things are centralized, but it makes sense that perhaps the current subset of the general public that is becoming more and more internet aware are now starting to become aware of the benefits of decentralization.

FlailFast• 12 years ago

A revolutionary bit of technology called the block chain

Nice overview, but something that bugs me whenever someone talks about Bitcoin (particularly some VCs): they talk about the blockchain independently, as a "technology," without recognizing that, in fact, it's as much a triumph of technology as it is a triumph of clever incentives. Small semantic point in this article, but more broadly applicable when folks say "they don't love Bitcoin, but they love the 'blockchain technology.'"

Spoiler alert: the blockchain is both reputable and trustless because there are thousands of miners securing the network by hashing the night away. And why do they do it? Because of the unit of account they generate by participating.

Whenever I hear someone talk about how they love "the technology of the blockchain" without explicitly mentioning the incentives at play, I automatically become suspect of whether they actually understand what's going on. (I'm looking at you, venture capitalists) Less of a problem with this article, but I think it's worth highlighting all the same.

yzzxy3• 12 years ago

I've also seen this behavior pattern a lot. I think there is a subtle line between disassociating yourself from evangelical Bitcoin fanatics and pushing a technology without really understanding it.

Also, isn't blockchain style computing obscenely inefficient? It would be great if someone well versed in these matters could weigh in.

VexXtreme• 12 years ago

It's only seemingly inefficient. It's basically the price you have to pay for maintaining a trustless distributed ledger.

If you were to run an organization/bank/Paypal clone that maintains a centralized ledger, it would be just as expensive (if not more), because you'd have to hire thousands of employees, compliance officers, lawyers etc. You probably see what I'm getting at.

There's always going to be a price for maintaining accurate accounting ledgers. The only questions is where those expenses are going to be pushed and whether it's going to be your payroll or your power bill.

jaekwon6• 12 years ago

There are two ways to secure the blockchain with a guaranteed level of security. Proof-of-work, and proof-of-stake. I use the second term very broadly, and I'm not referring to the consensus schemes that NXT or BitShares use. Anybody who says otherwise today regardless of the "marketcap" of their coin doesn't know what they're talking about from a CS/game-theory point of view, or they're ignoring the costs of proof-of-work schemes.

Proof-of-work is expensive and will ultimately be replaced with better consensus schemes that require no mining, namely those that only rely on cryptographic signatures, a good balance of incentives and penalties, and an underlying byzantine consensus algorithm.

Disclaimer: I'm one of a handful of people working on new consensus algorithms.

eric_bullington3• 12 years ago

I agree with this completely, and I agree with the peer comment that pinpoints the motivation for this stance to be these peoples' desire to disassociate themselves with the idea of Bitcoin and/or Bitcoiners.

Up to now, I've pretty much scoffed at this stance. The real breakthrough here is Bitcoin, or so I've always thought.

However, I've recently come across a project that made me realize that the blockchain technology may indeed bring about other revolutionary changes in addition to Bitcoin itself. One in particular is assembly.com and Coin, the blockchain-driven ownership records that formalize projects hatched under the assembly.com organizational framework. I'm most familiar with Assembly, but I wouldn't be surprised if other similar projects are actively underway.

If successful, assembly.com's approach to organizations has the potential to change the way we form companies, for the better. If you're interested in big societal changes brewing around us, I'd suggest going to take a look at how assembly.com works with the technologies that enable this type of organization (git and Github, in particular, in addition to the blockchain). If you've ever read the books of Visa-founder Dee Hock and his ideas about chaordic organizations, you'll immediately recognize this phenomenon. It's pretty fascinating, and as soon as I finish my current contract I intend to start working on an Assembly project.

But as for the power driving the blockchain technology (and thus enabling all of this), I agree completely that this is indeed Bitcoin and its underlying incentives. Pretty amazing stuff...

CMCDragonkai• 12 years ago

Assembly is really cool. But one problem ive had with it is how founding a project 600,000 coins for free, even for projects that aren't fully fleshed out. This makes it much more fruitful to start your own projects.

mrfusion• 12 years ago

Are you going to create a new assembly project or join an existing one?

TheOsiris• 12 years ago

I couldn't agree more. But I think many VCs use this line as an easy, soft no. There are always those who truly mean what they say when they say "we love blockchain technology but we don't believe in bitcoin," but those people don't know what they're talking about

Animats• 12 years ago

Well, in practice the Bitcoin block chain is now "secured" by about four centralized mining pools which control more than half the hash rate. Those pools operate big mining centers, many of which are in China.

Bitcoin mining is now less distributed than steel manufacturing. Home mining is now insignificant.

lumberjack• 12 years ago

Isn't there still a clear tendency towards centralization though with a handful of mining pools dominating the market.

It also has a problem of transparency in this respect because you cannot know whether two miners are actually just one miner manning two machines.

plainOldText2• 12 years ago

  "Most services today are centralized — sharing photos via social networks"

Yes, this service is centralized by virtue of choice. You can very easily share your photos via email, which is decentralized. Of course, you won't get all the features of a photo feed, but still, you can do it with no central authority.

Still, decentralized systems (in the context of the article) are great, but they present one little challenge: What incentives do people get in order to host a participating node/agent? In the case of bitcoin, the miners get a reward ($) each time they solve a new block, but unless all these decentralized systems leverage an existing blockchain/cryptocurrency system and have a system of rewards of some sort, they'll have a hard time springing into existence.

loup-vaillant2• 12 years ago

email, which is decentralized

Even email is centralised these days. Look how many providers host over 90% of people's primary email address.

plainOldText2• 12 years ago

The email, as a method of exchanging digital messages is not centralized. Now, it's true that a few providers host most people's email addresses, but there is not a single authority controlling the flow of the email messages.

If the case of a social network, you can't just switch and still remain connected with everyone else, because the provider controls the flow of your interactions. However, you can very easily switch your email provider, or even host your own, and as long as your friends still have the same emails, you're still "connected". I wouldn't call email centralized, but I agree is not decentralized 100 percent.

MichaelMoser123• 12 years ago

also facebook and friends got that big because they attracted some investment based on the promise to gain control of this mountain of personal data - all for the purpose of pushing personalized adds. Bruce Schneier means that when he says 'surveillance as the business model of the internet'

decentralized technologies would promise to benefit all participants; but i guess the problem here is legal protection; a registrar based on blockchain is a contract that would require legal recognition; how easy would it be to achieve such a change?

http://en.wikipedia.org/wiki/Legality_of_Bitcoin_by_country

this article has lots of words even when the currency is legal.

fidotron• 12 years ago

While preferring decentralization is relatively uncontroversial, the simple problem is the economic incentives to do it do not exist, and run very far in the opposite direction.

This isn't as simple as saying Google, Facebook etc. have a vested interest in promoting a centralized view, but that it's basically impossible to make money competing against them with a decentralized system.

There's a definite trend towards late 90s techno-utopianism appearing again.

dreamfactory23• 12 years ago

What about utility? Do people or businesses have an increasingly apparent incentive to own their data and online identity?

nine_k2• 12 years ago

While many other points may be valid, I'd say that mesh networks would be valuable in fringe situations (disaster relief, mass protests, etc) but cannot hope to replace the current Internet connectivity infrastructure.

Billions have been spent to create the IP transport networks. Building and operating them at current capacities, or even at 1% of current capacity, is of out of reach for most people who would run peer nodes of a mesh network.

Any node that'd happen to be in the bottleneck connecting two large segments, or just on a route to a very desired host, would be hit by a huge demand to pass traffic. Top-of-the-line 300 mbit wi-fi will be easily choked by that, not giving the node nearly enough bandwidth for its own requests. Huge fiber cables and microwave lines exist for a reason, and are not going to disappear.

The shape of the traffic they carry might change, though. E.g. with the advent of bittorrent, much of the traffic worldwide has become peer-to-peer / decentralized. This did not affect major carriers too much, except that they had to add bandwidth.

lifeisstillgood3• 12 years ago

I don't think that's the promise of mesh - it's more the local networks will share very small amounts of data relative to say a current peer1 but large compared to the usual daily browsing habits.

I guess it would be useful to gauge the size of local vs international traffic - Facebook does not need to send my data to SF unless I am in contact with someone in SF. Video uploads and downloads are similar. Most of my daily traffic almost certainly has a strong correlation to my physical location.

nine_k2• 12 years ago

If you need to send a bit of info to your physical neighbor, a mesh network might be a good solution. If you need to send a kilobyte to your friend half the world away, a mesh network would need to have a great number of hops.

If it just so happened that you are one of the few users with a mesh-network connection to a peer across the street, much or all of the traffic between your street and the next street will pass through you. Since it's a mesh network, you are expected to have a low-power, low-throughput node, which might be overwhelmed.

Do not conflate mesh networks (of nodes in physical proximity) with peer-to-peer networks (of nodes in logical proximity via all of the Internet).

forrestthewoods• 12 years ago

Hundreds of billions has been spent on smartphones. Trillions more will be spent on them in the following decades. It's something worth consideration.

ObviousScience2• 12 years ago

Point-to-point 1Gbit microwave equipment from Ubiquiti is within the price range of an avid consumer - about the same cost as a nice, but hobbyist grade 3D printer.

It's likely that anyone who wanted to operate a major server would be tied in to the network by one or two 1+gbps links, and probably colocating with other servers, allowing them to share the overhead of this, and have several connections operating to the network in parallel, lowering the stress on any one route.

Similarly, a lot of traffic is aggressively cached now to mitigate the need to hit the server with an identical request. There's no reason to think that certain kinds of content wouldn't be cached at various points in the mesh, making the total traffic to any particular server lower still.

A realistic scenario for a datacenter is that 50-100 people split the cost of having 10 1+gbps links to the network, hitting it at multiple points well spread in the network. The inside of the data center would do some routing to handle the load balancing. I expect the total cost of that would be about $35k to set up, but splitting it between that many people, each person is paying (on average, for 50 to 100 people) between $350 and $700 - well within a reasonable amount for a coop to set up a datacenter. Further, $35k is a small enough price tag that it would be acceptable for a business (or a couple businesses) to share such a datacenter themselves, if there were a particular incentive for them to operate as part of a community mesh, rather than with the current internet.

[1] http://www.amazon.com/Ubiquiti-AF24-Airfiber-Wireless-Comple...

[2] http://www.amazon.com/Ubiquiti-AF-5-airFiber-5GHz-Wireless/d...

lifeisstillgood3• 12 years ago

Tl;dr - three trends look like they will form a perfect storm - mesh networks, bitcoin/blockchain and Autonomous agents.

It's true - they do look like they will usher in a new world. It's pretty damn hard to justify the architecture of sending my wife's photos to SF just so they can appear on our neighbours Facebook feed.

But the technology feels too immature. I just wish I could tell if I would have felt the same way about perl cgi scripts in 94 - but I was too young then to know the difference.

sarahj• 12 years ago

I would argue that the gap between a centralized authority and a blockchain is a lot less that the gulf between a blockchain and a mesh network.

Mesh networks are awesome, they have a life of their own and an be split apart and stitched back together endlessly.

Blockchains, on the other-hand, are big and bulky, require an extraordinary amount of co-ordination between the maintainers and the miners and the clients to ensure everyone is running the right software. Many are so big that more people simply defer to a company to a handle their interactions with the chain - and as these companies grow they simple forgo the blockchain altogether in many instances and simply move bits about in their own database.

Also, can anyone offer an insight into the benefits of decentralized voting and storage? I can already decentralize who I rely on for storage by trusting different parties different amounts. There seems to be no benefit for everyone to trust the same blockchain for storage - and plenty of issues (hard forks etc.)

As for voting, the existence of a vote requires the existence of a group that wishes to vote (and a co-ordinated campaign.). Many countries have decentralized counts (e.g. each region votes separately) - and may separate organizations and individuals can become monitors to ensure the integrity of the ballot box and count. What then is the advantage of moving the actual vote to the blockchain?

haneefmubarak• 12 years ago

Personally, if we can find ways to create reliable, low latency mesh networks capable of transferring decent amounts of data, I would find that exciting.

With unreasonably recurring failures, there is little incentive for mainstream usage, as current networks are extremely reliable.

With excessive latency, there is little usability, as data may become outdated before being consumed, thus prohibiting real time applications (ie: one of the big reasons that we use computers).

With little bandwidth, little data can be sent, thus limiting the overall usefulness, especially in a day and age where data just keeps getting bigger.

But if you have all three, you can effectively have an internet that is extremely difficult to censor or alter, that is capable of connecting multitudes of users, that can handle most uses. And that, my friends, that will be the day.

diafygi• 12 years ago

Can pCells be used as a mesh?

swartkrans• 12 years ago

And governments can’t shut them down.

This is not true. Mesh networks are vulnerable to electronic warfare. Governments can use measures usually reserved for electronic warfare[1] to disrupt all signals except the ones the government needs. This is a standard capability of most modern militaries. Even the United States Marines, probably the least sophisticated among American branches of the military when it comes to intelligence and technology, have radio battalions that can do this.

[1] https://en.wikipedia.org/wiki/Electronic_warfare

jordigh• 12 years ago

Peer-to-peer wifi is old technology. It's called ad-hoc mode. Why has it never really taken off until iOS whatever version and Android whatever version?

p1mrx• 12 years ago

Follow the money. Android and iOS are developed by companies for whom decentralization is largely incompatible with their business models. There's no incentive to make it work smoothly.

orbifold2• 12 years ago

That's probably also the same reason why there isn't any local peer to peer file sharing software for phones. Sure it would not be quite as amazing as bittorrent, but at a college campus you probably could still get all the music and movies you want without any real fear of legal repercussions.

Nursie• 12 years ago

I've used it before to provide net access to two machines when there's only been one wire, and neither machine was able to run in AP mode. It was really, really awkward....

I think I used it once to connect a pair of PSPs to play a two player game....

I think the use-cases for true Ad-Hoc mode are actually pretty limited and in most cases one system (android or iOS these days) can probably run in AP mode anyway.

blorgle2• 12 years ago

Isn't this article just surface level technohype?

1. Mesh Networks: FireChat isn't a mesh network. There is a long thread on /r/Futurology about it, with lots of good posts describing what it is and the problems with implementing real mesh networks (https://www.reddit.com/r/Futurology/comments/2hvtoh/over_100...).

The article espouses all the benefits of mesh networks and glosses over the gritty facts of implementation that can't necessarily be solved by simply throwing more software at the problem.

2. The BlockChain: Yes, this is a massively distributed system. But the fact is it relies on the existence of a fast, reliable internet that so happens to be immensely centralised.

A rant I often like to have which makes me unpopular amongst the proponents of so called p2p systems is that the bootstrap of all of these so called decentralised systems is completely centralised. Bittorrent has trackers, Bitcoin uses DNS seeds to find peers, Tor uses directory servers and most others simply hardcode some URL list of peers into their source. If you can attack the bootstrap you can split these so called decentralised networks quite easily. Sybil attack is a common vector here.

The article also glosses over the fact that all the things which have driven bitcoin adoption and popularity (Blockchain.io, Mt Gox, BitStamp, BitPay, etc) are all centralised services which require the user to essentially give up any of the supposed benefits of decentralisation. Even in the case of mining, how many run their own bitcoind? 99% of miners run in a pool, another form of centralisation.

3. Autonomous agents: I can't even...

snowwrestler2• 12 years ago

Yes and this assertion is laughably wrong:

But history has proven that the centralized model is flawed and inefficient.

History shows the exact opposite. For example, at one point every business above a certain size had its own electric generator, and developed their own electricity on-site. In other words, decentralized power. That was replaced by contracts with central generation providers because that centralized model was so much more efficient.

You can see the same thing happening today with computing power (AWS, Azure, Google), music (Spotify, Pandors), movies (Netlfix), and books (Kindle).

AnthonyMouse• 12 years ago

The real answer is that size is a trade off. If you make entities too big you get all the problems of a monopoly, single point of failure, monoculture, etc. If you make entities too small you start having problems coordinating and knowing who to trust.

What you want is something in the middle. Email is a good example.

jefurii• 12 years ago

Yes and this assertion is laughably wrong: History shows the exact opposite. For example, at one point every business above a certain size had its own electric generator, and developed their own electricity on-site. In other words, decentralized power. That was replaced by contracts with central generation providers because that centralized model was so much more efficient.

Your comment assumes that efficiency is the most important thing. Addressing power imbalances and improving local resiliency are also important.

In the 1960s and 70s for many people computers meant centralized power and oppression. Computers were huge and expensive and therefore centralized, and only big government and big business (i.e. The Man) could afford them. A lot of the appeal of desktop computers in the 1980s was that they were personal: you could run the computing tasks that you wanted, on a machine that you controlled.

lebek• 12 years ago

Systems can be partly centralized and still benefit from being mostly decentralized. The examples you gave are good examples of this.

blorgle2• 12 years ago

I understand what you are saying, and as you said that is what we see today. The point I'm trying to make is that the utility of the system is driven by the centralised components. Without those centralised components, the so called decentralised systems have no utility to most users. What good is BitTorrent without TPB? How can you measure and transact in BTCUSD without going to BitStamp?

I'm not suggesting we couldn't get by on decentralised systems, but rather that such concepts are largely the domain of the geeky. Centralisation is what provides the convenience and trust that drives utility and adoption.

uptown• 12 years ago

Can anybody point me to additional sites/books/resources related to writing about the blockchain and decentralized research? It's a topic I'm very interested in, and something I'd like to have cross my radar more-often. Thanks in advance.

melling• 12 years ago

Hey, that's Gary, from Gary's Guide.

http://garysguide.com/events

He's sticking with his "Guy with the Red Tie" branding.

jeffreyrogers4• 12 years ago

The decentralized storage applications are interesting to me since I've been working on something similar for a networking project. The problem I see is data accessibility. What do you do when your data is shared among others laptops that aren't currently connected to the internet?

Clearly, for any sort of data where reliable access is important this won't be viable. It seems to me that it is only useful to do distributed storage among servers, rather than laptop or desktop computers, since servers have high availability requirements already and aren't likely to be taken off the network.

Second, you probably need to trust the servers you are storing your data on. Sure, you have some guise of anonymity, but ultimately you're probably transporting over TCP/IP and so you can look at the IP sender field to figure out who's data you're storing. Obviously this is a huge problem if some malicious person decides they want to corrupt or delete all files stored by certain entities.

Both of these problems (availability and integrity) seem largely intractable when dealing with distributed storage unless you make the limitations I suggested above (trusted computers and only using highly-available servers). Definitely an interesting thing to work on, but probably not viable for consumers (though I could see it working at an enterprise level).

emsimot• 12 years ago

What do you do when your data is shared among others laptops that aren't currently connected to the internet?

My understanding is that this can be mitigated by giving nodes a financial incentive to regularly be online and able to provide proof they still possess the data.

qwtel• 12 years ago

Second, you probably need to trust the servers you are storing your data on. Sure, you have some guise of anonymity, but ultimately you're probably transporting over TCP/IP and so you can look at the IP sender field to figure out who's data you're storing. Obviously this is a huge problem if some malicious person decides they want to corrupt or delete all files stored by certain entities.

Vitalik Buterin of ethereum worte an excellent blog post addressing all of these concerns. The conclusion is that you can in theory get 99.7% (99.99%) availability by using 3x (4x) redundancy from regular home users (defined as being reachable 50% of the time).

https://blog.ethereum.org/2014/08/16/secret-sharing-erasure-...

jeffreyrogers4• 12 years ago

I read the post, thanks, it was very interesting. It didn't address at all the concerns about a malicious user inspecting the IP sender field, however, but I suppose one could argue that because availability is high the network can incur the costs of malicious users as long as they do not get above a certain percentage of the entire user base.

Now, one problem with his math is this: the assumption is that any node that goes down will eventually come back up. However, this isn't the case. Say I get a new laptop and throw my old one out or otherwise wipe its drive. Well that's no longer part of the distributed network anymore. Of course, you can add to your file storage protocol by shifting files around as nodes go down, but that adds a lot of complexity.

Ultimately, the problems that are trying to be solved here are very difficult and I think a satisfactory solution via distributed computing is unlikely unless you are willing to make severe trade-offs (such as the ones I suggested, use trusted servers).

Now, making trade-offs doesn't make it useless. It just means the system isn't useful in all cases. Realistically, the data you have on your laptop probably isn't so important that it needs to be replicated across the globe, encrypted, and stored in such a way that the government can't access it. And if it is? Well, it's probably much simpler to just encrypt it on your end, send it to AWS, and get on with whatever you were doing that's so important.

jeffreyrogers4• 12 years ago

And just last week, a couple at Disney World had the first block chain marriage, recorded forever within the block chain!

So how does a blockchain divorce work? :)

orasis2• 12 years ago

I'm one of the pioneers of peer-to-peer file delivery from the late 90s. It turns out that decentralization is a hammer looking for a nail. Almost everything is simpler and more nimble when you centralize. The future espoused by this article ain't gonna happen.

jeffreyrogers4• 12 years ago

Yeah, this is definitely true. You can see the trend even in networking, where the routing protocols have typically been very decentralized in nature. Now with the advent of OpenFlow and data center networking we're starting to see a shift towards centralized control of network routing.

throwawayaway2• 12 years ago

file sharing? pretty big nail. a decentralised system supports easy access to files considered to be obscure or "long tail". unless you want to say that bittorrent is not a decentralised system.

orasis2• 12 years ago

Amazon S3 would be a much better choice for files considered obscure. A well designed decentralized system will have to spend a decent amount of effort to ensure that the long tail files don't drop off of the network. If you want fast access speed to those long tail files, now you're going to need even more redundancy to overcome access bottlenecks.

dreamfactory23• 12 years ago

What do you think of things like ostatus and technologies around http://www.w3.org/2005/Incubator/federatedsocialweb/wiki/SWA...?

There are a couple of considerations for me:

3rd party (particularly commercial entities) vs personal ownership of user data and identity seems like a nail which is quickly coming into focus a) with the ways that people are wanting to use social media and b) in light of recent privacy and monitoring revelations.

The kind of use cases above in SWAT, combined with p2p network effects seem to offer a glimmer of far more powerful functionality than any closed platform, however large, could ever keep up with (just as we see open source eat proprietary competitors through sheer scale and subsequent pace of innovation).

My instinct is that the federated technologies suffered from being ahead of their time and will get picked up again at some point.

cordite• 12 years ago

I was hoping for a discussion of things in the same nature of Diaspora (which is a somewhat decentralized social network which has faded in hype by now) or Aether, but I was rather turned off by just talking about the blockchain technology again.

090178• 12 years ago

http://www.peer5.com is right on this trend.

Could bring a lot of value around file sharing, streaming and such. Plug-in-less.

The demo page get to pitch in a couple of seconds.

https://peer5.com/downloader/

Have a look at it.

mslot• 12 years ago

Sounds cool an all, but mesh networks are slow and very insecure, the blockchain doesn't scale and thus cannot remain fully decentralized, and there's no practical way to run a trusted autonomous agent in an untrusted environment.

Computer science researchers have been working on and dreaming about these ideas for decades. It's great that some of them have found real-world applications, but that doesn't mean a revolution is near. It will be just as hard as it used to be.

Drabiv• 12 years ago

Currently, decentralized technologies seem to be quite futuristic, but not impossible. Benefits like better efficiency and resiliency over centralized technologies are substantial and clear. Adding onto Sherlock quote – [If you have something more efficient and] "When you have eliminated the impossible, whatever remains, however improbable, must be the truth" [and must happen]. The question remains – when this will happen?

The Imminent Decentralized Computing Revolution

Related Stories