Hashify: what becomes possible when one is able to store documents in URLs?

http://hashify.me/IyBIYXNoaWZ5CgpIYXNoaWZ5IGRvZXMgbm90IHNvbHZlIGEgcHJvYmxlbSwgaXQgcG9zZXMgYSBxdWVzdGlvbjogX3doYXQgYmVjb21lcyBwb3NzaWJsZSB3aGVuIG9uZSBpcyBhYmxlIHRvIHN0b3JlICoqZW50aXJlIGRvY3VtZW50cyoqIGluIFVSTHM/XwoKIyMgRG9jdW1lbnQg4oaUIFVSTAoKSGFzaGlmeSBpcyBkaWZmZXJlbnQgZnJvbSB2aXJ0dWFsbHkgZXZlcnkgb3RoZXIgc2l0ZSBvbiB0aGUgV2ViIGluIHRoYXQgKipldmVyeSBVUkwgY29udGFpbnMgdGhlIGNvbXBsZXRlIGNvbnRlbnRzIG9mIHRoZSBwYWdlKiouCgpUaGUgYWRkcmVzcyBiYXIgdXBkYXRlcyB3aXRoIGVhY2gga2V5c3Ryb2tlIGFzIG9uZSB0eXBlcyBpbnRvIHRoZSBlZGl0b3IuCgojIyMgQmFzZTY0IGVuY29kaW5nCgpPbmx5IGEgdGlueSBmcmFjdGlvbiBvZiBhbGwgVW5pY29kZSBjaGFyYWN0ZXJzIGFyZSBhbGxvd2VkIHVuZXNjYXBlZCBpbiBhIFVSTC4gSGFzaGlmeSB1c2VzIFtCYXNlNjRdWzFdIGVuY29kaW5nIHRvIGNvbnZlcnQgVW5pY29kZSBpbnB1dCB0byBBU0NJSSBvdXRwdXQgc2FmZSBmb3IgaW5jbHVzaW9uIGluIFVSTHMuCgpUaGlzIHRyYW5zbGF0aW9uIGlzIGEgdHdvLXN0ZXAgcHJvY2VzczogW1VuaWNvZGUgdG8gVVRGLTggY29udmVyc2lvbl1bMl0gYXMgb3V0bGluZWQgYnkgSm9oYW4gU3VuZHN0csO2bSwgZm9sbG93ZWQgYnkgYmluYXJ5IHRvIEFTQ0lJIGNvbnZlcnNpb24gdmlhIFtgd2luZG93LmJ0b2FgXVszXS4KCiMjIyMgRW5jb2RpbmcKCiAgICA+IHVuZXNjYXBlKGVuY29kZVVSSUNvbXBvbmVudCgnw6dhIHZhPycpKQogICAgIsODwqdhIHZhPyIKICAgID4gYnRvYSh1bmVzY2FwZShlbmNvZGVVUklDb21wb25lbnQoJ8OnYSB2YT8nKSkpCiAgICAidzZkaElIWmhQdz09IgoKIyMjIyBEZWNvZGluZwoKICAgID4gYXRvYigndzZkaElIWmhQdz09JykKICAgICLDg8KnYSB2YT8iCiAgICA+IGRlY29kZVVSSUNvbXBvbmVudChlc2NhcGUoYXRvYigndzZkaElIWmhQdz09JykpKQogICAgIsOnYSB2YT8iCgojIyBVUkwgc2hvcnRlbmluZwoKU3RvcmluZyBhIGRvY3VtZW50IGluIGEgVVJMIGlzIG5pZnR5LCBidXQgbm90IHRlcnJpYmx5IHByYWN0aWNhbC4gSGFzaGlmeSB1c2VzIHRoZSBbYml0Lmx5IEFQSV1bNF0gdG8gc2hvcnRlbiBVUkxzIGZyb20gYXMgbWFueSBhcyAzMCwwMDAgY2hhcmFjdGVycyB0byBqdXN0IDIwIG9yIHNvLiBJbiBlc3NlbmNlLCBiaXQubHkgYWN0cyBhcyBhIGRvY3VtZW50IHN0b3JlIQoKIyMjIFVSTCBsZW5ndGggbGltaXQKCldoaWxlIHRoZSBIVFRQIHNwZWNpZmljYXRpb24gZG9lcyBub3QgZGVmaW5lIGFuIHVwcGVyIGxpbWl0IG9uIHRoZSBsZW5ndGggb2YgYSBVUkwgdGhhdCBhIHVzZXIgYWdlbnQgc2hvdWxkIGFjY2VwdCwgYml0Lmx5IGltcG9zZXMgYSAyMDQ4LWNoYXJhY3RlciBsaW1pdC4gVGhpcyBpcyBzdWZmaWNpZW50IGluIHRoZSBtYWpvcml0eSBvZiBjYXNlcy4KCkZvciBsb2

by potomak • 15 years ago

186 77 15 years ago

hashify.me

cobychapple• 15 years ago

What becomes possible? The entire internet could effectively get rid of hosting account providers, with each page in every site being contained in a hashify URL, and with each page linking to other pages using other hashify URLs.

Trouble is, there might be a DNS-like system needed to match hashify URLs to more human-readable strings (or a way for existing DNS to resolve to hashify style URLs).

Neat idea.

vidarh• 15 years ago

The data needs to be stored somewhere. In their implementation they in effect use bit.ly as the hosting provider for the data by shortening the url's, so while it's a fun little experiment, it boils down to a content addressable system. We already have good examples of content addressable systems. Git for example is built on content addressable storage.

djhworld2• 15 years ago

It also assumes that there's no limit on URL space that bit.ly provide. Tomorrow they could just max out the "long_url" field or whatever they call it to just accept 1500 chars or something

friggeri• 15 years ago

The real trouble is that when you link to a hashified URL, you are actually embedding in your web page (an encoding of) the content of the page you are linking to. Think matryoshka.

LeafStorm• 15 years ago

Not to mention that makes link cycles impossible.

austintaylor• 15 years ago

Right. Even without link cycles, any link would have to contain (nearly) the whole internet.

seanp2k24• 15 years ago

in essence, this would be moving away from a model of "large networks of connected pages/sites" to "a large amount of single documents with no meaningful mechanism of inter-connectedness".

Think about this like a PDF where stuff is embedded instead of in separate files.

mattvot3• 15 years ago

Boiling it down, it's a new file format with a built in viewer. You need to find a way to store the data.

Interesting, but I can't think of any practical application, apart from the service provider not having to worry about storage (maybe that's key ... more thinking needed).

irrumator• 15 years ago

It would be except it's not new, it's a dupe of post from not too long ago that had a large amount of points: http://news.ycombinator.com/item?id=2464213

juanre• 15 years ago

Very neat idea, but I think the reliance in bit.ly is self-defeating. This kind of approach would allow people to distribute documents using the web without having to trust them to a particular server, which can be very convenient if your target audience is in a country where access to the server storing your documents can be closed. For this to work you need to be able to recover the document from the URL locally.

Some years ago a friend and I wrote http://notamap.com, a very similar idea for sharing/storing/embed geotagged notes fully encoded on a URL, without having to rely on a server. Looking at it now I wish we had not put all the crazy animations. Maybe I should recover it and simplify the UI.

rapala• 15 years ago

I just can't see the gain here. You need a server to distribute the URLs in any case. You are just moving the data from the server that served the document to the server that servers the URLs. It is still the same data, just in different form.

For this to work you need to be able to recover the document from the URL locally.

How about saving the document?

seanp2k24• 15 years ago

Right, exactly. It's only really cool for magnet links because the law is different for "linking to content" vs "hosting content" right now.

One step further in this direction would just mean "including all the data for one or more documents on one page". You've just invented...a document, possibly with more than one media type.

Think about it this way: You have a 10K document which contains 200 bytes of link data; a URI to another hypertext document that is 5K in size...vs...you have a 15K document which contains 5K of "link data"; the other document.

zalew2• 15 years ago

The first use case that comes to mind is anonimity.

mattvot3• 15 years ago

Solutions like pastebin.com provide the same, with a small url.

tkellogg4• 15 years ago

These messages are SOPA proof. They can never be "taken down" since they don't actually reside on the server

Angostura2• 15 years ago

"Internet Explorer cannot display the webpage" is what happens here (IE 8).

tkellogg4• 15 years ago

Not really a good reply, but I think that hashify.me's potential for an IE audience was probably small to start with. But consider this: if this idea took off, wouldn't this press MS into keeping IE more modern?

brudgers• 15 years ago

My mobile browser of choice does not support cross browser resource sharing, according to the article...or rather the error message I get in lieu of the article.

Jare• 15 years ago

The original version of Mr Doob's GLSL Sandbox at http://mrdoob.com/projects/glsl_sandbox/ used the same approach, but increased the maximum possible size of the document by doing LZMA compression before base64.

The project later moved to http://glsl.heroku.com/ with an app-driven gallery, and that particular feature went away. I think that is a pretty natural evolution of any such idea, so I'm not convinced of hashify's logevity, but hey, simple sometimes is really enough.

dools• 15 years ago

I took a similar approach to this with http://cueyoutube.com and recently found snapbird which gives extended twitter search capabilities. So the URL contains the playlist and twitter becomes the data base, so I just tweet my playlists and they're "saved". You can see all the lists I've created by searching the account iaindooley and search term cueyoutube in snapbird.

rsaarelm2• 15 years ago

I've sometimes wondered about a system where the URL of a document is an actual hash, like SHA-1, of the document. That'd chance the semantics of hyperlinks from "link to document at this internet address" to "link to document with these contents", just like Hashify does, but it could do arbitrarily large documents.

The tricky part with that system would be that you'd also need some new mechanism to retrieve the files. Instead of the regular WWW stack, you'd need something like a massive distributed hash table that could handle massive distributed querying and transferring the hashed files. Many P2P file sharing systems are already doing this, but a sparse collection of end-user machines containing a few hashed files each isn't a very efficient service cloud. If every ISP had this sort of thing in their service stack or if Amazon and Google decided to run the service, all of them dynamically caching documents in greater demand in more nodes, things might look very different.

This would mean that very old hypertext documents would still be trivially readable with working links, as long as a few copies of the page documents were still hashed somewhere, even if the original hosting servers were long gone. It would also make it easy to do distributed page caching, so that pages that get a sudden large influx of traffic wouldn't create massive load on a single server.

On the other hand, any sort of news sites where the contents of the URL are expected to change wouldn't work, nor would URLs expected to point to a latest version of a document instead of the one at the time of linking. Once the hash URL was out, no revision to the hashed document visible from following the URL would be possible without some additional protocol layer. The URL strings would also be opaque to humans and too long and random to be committed to memory or typed by hand. The web would probably need to be somehow split into human-readable URLs for dynamic pages and hash URLs for the static pieces of content served by those pages.

I'm probably reinventing the wheel here, and someone's already worked out a more thought out version of this idea.

mhitza2• 15 years ago

http://en.wikipedia.org/wiki/Uniform_resource_name

rsaarelm2• 15 years ago

Right. Turns out my use of 'URL' everywhere in grandparent comment is a misnomer then. Should've used URN or URI.

I'm not quite sure if URN is exactly right for the hash thing either, given that it both fails to unify things which humans would probably assign the same URN to, such as two image files of the same picture using different encodings, and it has the theoretical chance of assigning the same hash to two entirely different things.

sp3323• 15 years ago

I think Freenet already does this https://en.wikipedia.org/wiki/Freenet#Keys Edit: I should point out that it's a separate network from "the web".

ivank• 15 years ago

http://www.ccnx.org/about/

"A New Way to look at Networking" http://www.youtube.com/watch?v=8Z685OF-PS8

pixelcort• 15 years ago

Check out http://en.wikipedia.org/wiki/Magnet_URI_scheme

In a single link for a file, it can contain multiple hashes for multiple means of retrieval.

eternalban3• 15 years ago

> I've sometimes wondered about a system where the URL of a document is an actual hash, like SHA-1, of the document

Git.

It may be of interest to view this duality as an analog to the duality of location addressing (iterative) vs value addressing (functional) in context of memory mangers. The general (hand wavy as of now) idea is a distributed memory system with a functional front-end (e.g. Scala/Haskell).

antimatter15• 15 years ago

The creator of Freenet made something called dijjer, which mirrors http files in a p2p network accessible by prepending http://dijjer.org/get/ to it. But it looks like he's no longer maintaining it.

http://code.google.com/p/dijjer/

ColdAsIce• 15 years ago

Fuckify this ify trend to emulate or ride the spotify (anti)fame.

Is it just me or does anyone else also just back away whenever there is a project which turns nouns into verbs with ify? Spotify is a sockpuppet of

sp3323• 15 years ago

-ify is a pretty common suffix in English. It means to turn something into something else. https://en.wiktionary.org/wiki/-ify and https://en.wiktionary.org/wiki/Category:English_words_suffix...

7952• 15 years ago

Maybe this would work well in an email? Especially if you want to get content past filtering.

icebraining• 15 years ago

Yes, but so would any pastebin website. You could say the advantage is that the content is not available to the server (since its transferred in the URL itself), but then it is when you actually read it, so it's not any more private.

aurelianito3• 15 years ago

I don't get it. Why do they claim that this is in any way better than a data:// url? (http://es.wikipedia.org/wiki/Data:_URL)

sp3323• 15 years ago

This lets you use the data as a piece of a URL, so you can pass it as a CGI query string to another web page.

aurelianito3• 15 years ago

I still don't get it. Passing data as pieces of URLs is what normal parameters are for and data URLs are for generating a "virtual file", id est a link that contains all the information of the file linked.

With those 2 things, everything should be covered.

ams6110• 15 years ago

In English: http://en.wikipedia.org/wiki/Data_URI_scheme

tony_le_montana• 15 years ago

Great idea this. But it saves on each edit and likely to hit rate limit on bit.ly :(

sgdesign• 15 years ago

I really like this approach, in fact that's what I used for http://www.patternify.com/

This way the whole tool can be 100% client-side javascript, without a need for any back-end.

jroseattle• 15 years ago

This is cool, but I wouldn't use it for any real documents. I care about versioning, edit history, etc.

orclev• 15 years ago

This is an ancient idea. I read a 2600 article back in the early 2000s or possibly late 1990s that did essentially this same thing using a bash script and one of the first URL shortening services available at the time.

eternalban3• 15 years ago

   What has been will be again,
   what has been done will be done again;
   there is nothing new under the sun.

   - Ecclesiates 1:9

seanp2k24• 15 years ago

...and the general concept of "embedding one type of data inside another" predates computers and modern civilization.

mmahemoff• 15 years ago

I looked at URL shortener limits some time ago and found these approximate limits by trial-and-error:

* TinyURL 65,536 characters and probably more, but requests timed out; there isn’t an explicit limit apparently

* Bit.ly 2000 characters.

* Is.Gd 2000 characters.

* Twurl.nl 255 characters.

This was 2.5 years ago, not sure how much of these have changed (other than bit.ly, which the linked article confirms is 2048, probably the same as when I tested it).

http://softwareas.com/the-url-shortener-as-a-cloud-database

eli• 15 years ago

2000 is roughly the maximum length of a URL that IE can handle, incidentally.

michaelfeathers• 15 years ago

Well, for one thing, you can't tweet a link.

samgranger3• 15 years ago

Link shortening service?! Twitter automatically shortens it for you :)

nerfhammer3• 15 years ago

ought to gzip the string before it goes to base64 while you're at it

gresrun• 15 years ago

We actually do this exact thing to send dynamic parameters to a chart-generating backend server. It works great; you get a surprising amount of compression using gzip (2-4x space savings) and the URLs are naturally cached by proxies without any magic!

nerfhammer3• 15 years ago

If it's mostly ascii compression is something you should almost automatically think about.

There's also snappy http://code.google.com/p/snappy/

GICodeWarrior• 15 years ago

http://www.semicomplete.com/projects/tastydrive/

vidoss• 15 years ago

This is very cool. Not the tasydrive, your "keynav"

http://www.semicomplete.com/projects/keynav/

djbender• 15 years ago

Older Hacker News Post: http://news.ycombinator.com/item?id=2464213

samgranger3• 15 years ago

Cool to see but stupid idea, who in their right mind would use this for production?! By using such a "technology", you lose SEO strength due to urls-not-being-like-this.html and even worse, what can stop me from publishing a fake press release on there site/spamming porn and getting that URL indexed? And what are the benefits? To also bring SOPA into this, couldn't I share copyrighted material on someone's site like this? How could they control that?! Besides blocking each URL manually. Just seems dumb. As a concept, cool, but for production.... Yikes?!

samgranger3• 15 years ago

Obviously you could also have a database with all content URL strings that you publish - but that makes this technology worth nothing at all.

feralchimp• 15 years ago

Clever? Yes.

But URL shortening services are a public good, and hacking one to be your personal cloud storage platform is kind of a dick move.

jheriko• 15 years ago

agreed

choffstein• 15 years ago

Repost: http://news.ycombinator.com/item?id=2464213

d2fn• 15 years ago

some observations: 1. when content changes every hyperlink to that content must change along with it. 2. pass by reference (url) is no longer possible.

tronicron• 15 years ago

Damn, so cool!

markkum• 15 years ago

Check out https://neko.io/ ... we are scrambling/encrypting messages into URLs which you can then share on Facebook, Twitter or where-ever.

Hashify: what becomes possible when one is able to store documents in URLs?

Related Stories