Dropbox: the new file system of the web

http://collaborable.com/blog/dropbox-the-new-file-system-of-the-web

by ericingram • 15 years ago

98 60 15 years ago

mattberg• 15 years ago

giving the user that much control, using his example of a spreadsheet on his computer, makes me cringe thinking about how much the user could mess things up with the data.

ericingram14• 15 years ago

By contrast, it makes me excited thinking about the possible solutions to the problems you just described. What could be done? Maybe there is a revision system in the web app that shows the changes which must be confirmed before applying? This also would give you web app the ability to easily version control data.

Awesome

molecule• 15 years ago

"How do I undelete files or recover old versions of files?"

http://www.dropbox.com/help/11

daniel_levine2• 15 years ago

Just like with the web application developers will just need to control what a user can do. Users can break native client apps by screwing with config files now but for the most part they don't I think as long as a developer creates some friction (like hidden files) it won't be too much of a problem.

daniel_levine2• 15 years ago

I think this is right on. Dropbox is a virtual file system and not just for the web also for offline. And it works on every platform, I'd much prefer to have things like my contact book and email on Dropbox than where they are today.

ericingram14• 15 years ago

I'm guessing the fact that it 'works on every platform' has a lot to due with it's past and future success.

It's a jump back to where we came from in a sense. The desktop file system works well for a lot of business people, which is partly why Dropbox is growing so fast.

bprater• 15 years ago

Dropbox has now officially replaced "My Documents" on all my computers, Windows and Linux. I recently reinstalled Windows 7 on my gaming PC. When it finished up, I installed DropBox, logged in to my account and went to watch a movie. A couple hours later, I was synced up. Too simple.

So, yes, I can't wait for more web apps to allow me to save my data to my Dropbox!

ericingram14• 15 years ago

I'm building one

keithpeter• 15 years ago

OK, at work we use Sharepoint to collaboratively edit and push out data to colleagues as Word and Excel files.

Will your app let me do the same kind of thing at home with outside work friends and the voluntary work I do?

lawnchair_larry• 15 years ago

The security implications (which dropbox do not take seriously) of this are a nightmare. I feel sorry for all of the network admins and security people who have to deal with the fact that all of their employees are unintentionally exfiltrating everything.

gordonc• 15 years ago

We at http://views.fm completely agree

ericingram14• 15 years ago

Awesome, this feels like a very good idea. But you request full access to sign up? Why not just an app folder?

clemesha• 15 years ago

Cool! What about S3 bucket support?

trotsky2• 15 years ago

Looks like your TOS grants you a perpetual all rights license to all content in the user's dropbox without any of the usual qualifications about only as far as it's needed to provide the service?

views.fm: By providing any content to this web site: (a) you agree to grant the site editor a worldwide, royalty-free, perpetual, non-exclusive right and license (including any moral rights or other necessary rights.) to use, display, reproduce, modify, adapt, publish, distribute, perform, promote, archive, translate, and to create derivative works and compilations, in whole or in part. Such license will apply with respect to any form, media, technology already known at the time of provision or developed subsequently; 

Dropbox: By using our Services you provide us with information, files, and folders that you submit to Dropbox (together, “your stuff”). You retain full ownership to your stuff. We don’t claim any ownership to any of it. These Terms do not grant us any rights to your stuff or intellectual property except for the limited rights that are needed to run the Services, as explained below.

We may need your permission to do things you ask us to do with your stuff, for example, hosting your files, or sharing them at your direction. This includes product features visible to you, for example, image thumbnails or document previews. It also includes design choices we make to technically administer our Services, for example, how we redundantly backup data to keep it safe. You give us the permissions we need to do those things solely to provide the Services. This permission also extends to trusted third parties we work with to provide the Services, for example Amazon, which provides our storage space (again, only to provide the Services).

I assume that's a cut and paste job gone horribly wrong, but that is pretty gross.

rhizome2• 15 years ago

I would imagine that a storage company would have something like that if they planned public/social features, such as an imgur for pix you're already storing with them.

collint• 15 years ago

This is not how the Dropbox API works at this moment.

There is no mechanism to push updates to your application. Only the desktop client gets async updates. It will be a glorious day when such an API exists.

But don't hold your breath, it's been a topic on their forums for years.

ericingram14• 15 years ago

Push events to the web app isn't important to me. Syncronization with a web app every few minutes would work well, and you could login to push a button if you needed the changes pulled immediately.

chubs• 15 years ago

That's the way i implemented it in an iphone app i recently made: a 'sync now' button to pull anything new and push any updates to the server. Works quite well - I recently deleted my app (iphone died) and reinstalled it, one 'sync now' later and all my data was back.

I'm a big fan of using dropbox like this, and am hoping to explore this approach in a lot of productivity apps i plan to make in the future. I like the idea that the user can 'see' their data in a readable format (json,csv,yaml - i still haven't decided which i prefer!) in their dropbox\apps\MyApp folder.

It's like icloud but you feel in control, rather than 'my data is out there somewhere but i can't see it'.

drivebyacct2• 15 years ago

In terms of these sorts of developer oriented features, I'm much more excited to see Camlistore take off. Portable, better sync options, open source, support for any backend, etc. Just something to look at if this topic is interesting.

In fact, their semi-slogan is "your home directory for the web". Very privacy focused (an encryption layer for backends is in progress based on their issues log), since that seems to get buzz these days.

http://camlistore.org/

ericingram14• 15 years ago

Camlistore looks really interesting

peteforde3• 15 years ago

I understand and in some ways share your enthusiasm. I think Dropbox could be the file I/O layer of the social web... if only they would publish a proper events API.

As it stands, the only way to receive updates is to poll for changes. This is a kludge at best (and takes away from the magic in a significant way in practice) and completely unrealistic at scale. That is, if I have a million users and I want to update every 15 seconds, Dropbox is going to (rightfully) shut my IP down.

If developers could define WebHooks to receive events, that'd be a great start. I'd happily pay for that.

If developers could subscribe to a WebSockets or event XMPP notification stream of updates, that'd be freaking amazing. I'd stand in line to pay for that. And that's when you'd see an explosion of applications (web, mobile and desktop) implementing Dropbox as their filesystem.

We've been asking nicely for years. If there's anyone here with pull or influence at Dropbox, please give us devs some real-time love. Otherwise, please add your voice to the choir:

http://forums.dropbox.com/topic.php?id=20226&replies=52#...

ericingram14• 15 years ago

I could build a Dropbox notification app in an afternoon or two, and sell it to you and a few thousand friends for $5/mo :)

peteforde3• 15 years ago

Eric, ideas are cheap and execution is hard. I happen to believe that you're completely out to lunch if you think you could build this in a practical, scalable way that is useful in the ways I describe above.

However, I'm open minded and you seem confident — so if you believe that you're sitting on something, please feel free to share how you'd structure such a service. If you get it right, I'd pay you a lot more than $5, as long as it had the blessing of Dropbox (ie. doesn't get API throttled and isn't going to get cut after I integrate with your tool).

LeafStorm2• 15 years ago

Or even setting a callback URL for a file or folder that POSTs some JSON when the file changes. Simpler for everyone.

kelnos• 15 years ago

Parent addressed that: that's what a WebHook is.

stanleydrew• 15 years ago

That's what this means:

"If developers could define WebHooks to receive events, that'd be a great start. I'd happily pay for that."

sleet• 15 years ago

You can vote for it here: https://www.dropbox.com/votebox/5972/events-api

brlewis4• 15 years ago

I too am hoping for an improvement that will scale to a million users. But here's what I do that should scale to tens of thousands decently:

I run dropboxd on my web server. Instead of using the API, users share a folder with my app. Dropboxd gets the stream of changes, and my app polls the local filesystem for changes instead of polling across the net.

ericingram14• 15 years ago

Whoa, that's really smart. Going to try that!

peteforde3• 15 years ago

How do you handle quotas?

trotsky2• 15 years ago

Drag and Drop file uploads/downloads in HTML5: http://tutorialzine.com/2011/09/html5-file-upload-jquery-php...

WebDAV mounts as a network share out of the box in Win/OSX/Lin

I love dropbox but I could see it being much worse than usual for a fair % of users

freshhawk2• 15 years ago

I wonder if anyone has the ability to understand and use dropbox this way but isn't able to mount network shares. I feel like anyone who would use this would see it as worse than usual. Dropbox and it's clones are great, and it's good to see this convenience available to a more mainstream audience.

But as someone who has used sshfs for years to get seamless access to all my boxes from wherever I am I'm certainly not getting excited about some kind of crippled filesystem-over-http fad. It took mobile to remind people that the internet is good for more than just the web but I don't see that trend stopping (even if we do have to start calling software "native apps")

The web's great, but this everything-in-the-browser silliness means people are using crappier tools than they used to have. I find it annoying but practically it's just not a stable state of affairs because it's so easy to change.

ericingram14• 15 years ago

This seems to conflict: "The web's great, but this everything-in-the-browser silliness means people are using crappier tools than they used to have."

... with your sentiment on the original post; that it was pointing out how Dropbox integration lets people get out of the browser and use the tools they might be more comfortable with...

Are you for, or against APIs to connect web and desktop and mobile?

Shorel2• 15 years ago

I believe the issue is that web development is normally far easier than the alternatives.

So, some ideas are tested in the web, and if they are successful and they benefit from going native, they will be reimplemented in a more close to the metal way. Or not.

GoogleDocs -> Dropbox

VVVVVVV in flash -> VVVVVVVV in C

Some future game in WebGL at 5 FPS -> Same native game at 120FPS with anti-aliasing and anisotropic filtering.

OTOH: Mail clients -> Web mail

Someone• 15 years ago

AFAIK, Dropbox is a 'filesystem' that does not support partial file overwrites, does not do locking in a meaningful way, etc.

Because of that, anything built along the lines outlined in this post will be brittle. For example, if one updates that 'products.csv', one must be absolutely sure that one applies those updates to the last version saved. A solid tool would have to detect races and offer merging of the files. I would rather use some SQL in the cloud or, lacking that, some SCM. Those, at least, come with tools to manually/automatically handle such race conditions.

pavpanchekha• 15 years ago

To be fair, opportunistic locking is implemented in the API (you can pass a revision ID and the call fails if this is not the current version, or something like that).

talaketu• 15 years ago

That would be optimistic locking.

mvanveen• 15 years ago

I really like Dropbox. In fact, it's fair to say I love Dropbox. It's made so many use cases so much simpler. However, I am f*g sick of this perception that Dropbox is the silver bullet for every storage solution. This sort of sensationalism is just stupid.

Dropbox is a cloud-hosted, distributed system and just like any other large, distributed system, it's subject to some very real constraints in terms of latency and reliability.

In other words, it's not fair to claim that Dropbox is some magic silver bullet that will solve every scaling problem you're subject to, act like a rock solid filesystem which behaves identically to the one you have locally, or solve your love life problems.

Maintaining this perception is just going to create frustration for people who expect it to act this way, or worse, create a living hell for the developers maintaining the glue code between your system and the Dropbox API.

You try figuring out what the fuck is going on with your system when the "spooky" or "weird" version regressions start occurring. The arms will raise in a giant, collective shrug. Watch in horror as your http requests timeout because the roundtrip time between your box and Dropbox is that bad. The sodden faces of your product customers and the million "(so-and-so's conflicted copy of " files laying around your filesystem which JUST. KEEP. COMING. BACK. will be enough to make you wish you never drank the kool aid.

Now, I'm not saying that the Dropbox REST API isn't the best thing since sliced bread. All I'm saying is it has it's purposes and there are use cases and times when it simply isn't an appropriate choice.

In short, please respect the fact that distributed storage is a hard problem with hard constraints, stop proselytizing Dropbox as the 2nd coming, and relax and enjoy your backed up media and documents on your iPad already. Seesh.

ericingram14• 15 years ago

You're right in the sense that the pattern is to be excited about, and not only Dropbox itself.

pak• 15 years ago

I'm with you; I've already experienced various spooky things with desktop applications on top of Dropbox (like the excellent Papers2 by mekentosj) because of file conflicts and latency. And that's not Papers2's fault, since it would have to be redesigned to be tolerant of such things. But it goes a long way toward showing that Dropbox is not a syncing FS that can put any local application data in the cloud. In particular, Dropbox is bad for typical database use patterns (simultaneous writes, transactions) and the versioning and locking are not tunable.

wavephorm• 15 years ago

Dropbox is merely providing the blueprint for how a cloud a file system should work. I hope they're using their funding to develop something completely above and beyond what they're currently offering, because the network storage space is going to become awfully crowded in a few years.

ericingram14• 15 years ago

"...merely providing the blueprint for how a cloud a file system should work."

That sounds like kind of a big deal to me, but hey call me an optimist.

JoshTriplett• 15 years ago

This seriously disturbs me. Dropbox has several fundamental flaws that make it entirely unsuitable for widespread adoption as anything other than a personal file store for people who don't care about privacy. A "filesystem of the web" would need security, encryption, fine-grained sharing controls, reliability, and numerous other features. Meanwhile, the whole sync mechanism of Dropbox doesn't really matter for a "filesystem of the web", because changes can just occur directly in the remote filesystem rather than locally with syncing.

Now, if someone called S3 "the new filesystem of the web", that would make more sense. Numerous services build on S3, including Dropbox itself, and it provides enough tools and API to allow those services to include all the critical features mentioned above. Add a few features for users to hook up their own S3 data to services rather than the S3 accounts of those services, and you'd have something even more interesting. (The same thing applies to any equivalent service to S3, and many such services exist.)

extension• 15 years ago

Meanwhile, the whole sync mechanism of Dropbox doesn't really matter for a "filesystem of the web", because changes can just occur directly in the remote filesystem rather than locally with syncing

Excel can't make changes directly to the remote filesystem. That's the novelty of this scheme. It isn't yet another cloud database, it's a way to share local files with cloud apps.

I have mixed feelings about this.

It could serve to bring data out of the silos, and back under user control. That would be an epic win.

But it's not federated. And I anticipate problems with access control, which needs to be per app, per file, and separate read/write.

Aside: I recently had a nasty scare when a shitty iOS password manager deleted my Keepass database from my Dropbox. If hadn't found a copy in the Dropbox cache on my laptop, I would have been in a world of shit.

brlewis4• 15 years ago

I'm writing this reply from a commuter train with wifi that goes away whenever we enter a tunnel. My point of view is that a filesystem of the web has to support disconnected operation. The Coda filesystem has worked that way for decades.

Dropbox: the new file system of the web

Related Stories