Bypass the internet for large transfers: AWS Import/Export

http://aws.amazon.com/importexport/

by mattjaynes • 17 years ago

86 30 17 years ago

aws.amazon.com

wheels• 17 years ago

Obligatory Tanenbaum quote:

"Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway."

jeffbarr• 17 years ago

Indeed, so obligatory that I included it in my blog post:

http://aws.typepad.com/aws/2009/05/send-us-that-data.html

jimbokun• 17 years ago

Kudos (to all of you at Amazon) for being the first to turn that quote into a product feature. Certainly seems obvious in retrospect.

jjs• 17 years ago

Sometimes a big truck has more bandwidth than a series of tubes...

eugenejen• 17 years ago

Bandwidth and latency are two different things. But it seems we are so often confused by these two concepts.

I feel funny because Werner Vogels is Andy Tannenbaum's phd student. That's how a concept propagates from one generation to next.

tophat02• 17 years ago

Terrible latency, though.

alexkay2• 17 years ago

Dupe: http://news.ycombinator.com/item?id=620089

RiderOfGiraffes• 17 years ago

Although this one is getting the comments.

boundlessdreamz• 17 years ago

Probably because of a better title giving people an idea about the service even before they read the article.

alexkay2• 17 years ago

The other earlier submission has been deleted and I'm being downmodded. Fair enough, I guess I won't be reporting duplicates any more.

drawkbox• 17 years ago

New cloud feature: Pony Express

Only kidding this is great for immense datasets.

stevejalim• 17 years ago

<lamejoke> Presumably a feature for Django devs</lamejoke>

dfield• 17 years ago

Google had a neat feature related to this in implementation but not goal since 2007ish. It was called Google Datasets and let you mail in a hard drive. Google would then host the data on the web. It's audience was purely academic, though... no money exchange involved.

Anyway, it got cut back last Fall. Cool to see it's still living on in some form.

danw• 17 years ago

Oh that's a shame, I didn't know it had been cut.

Amazon has a similar scheme "Public Datasets", that store data as an EBS rather than an S3 bucket http://aws.amazon.com/publicdatasets/

katamole• 17 years ago

Not really "AWS" then!

pierrefar• 17 years ago

How do they do virus and security checking? I don't know how they import the data, but it could be that they use a system that can be exploited using a security vulnerability?

It's a (very) long shot for this kind of thing to work, but still it's a risk worth asking about. They are at the end of they day hooking up hardware to their computer - i.e. physical access to the machine.

cracki• 17 years ago

would you risk your account by intentionally adding malware to your data sets?

HDDs are passive, there is no danger in merely hooking one up. and as long as the file system drivers are solid, shoveling these bits into "buckets" (whatever these are) is safe too. you're just shoveling the bits, not looking at them.

this is about data, not executable code.

loop• 17 years ago

They only support SATA and USB-2, which unlike Firewire, do not work by DMA, IIRC. I'd assume they do not support Firewire for exactly this reason: to prevent a device from reading the systems memory directly (known as pwn-by-iPod)

mtkd• 17 years ago

When I read the announcement this morning, it occurred to me that the reason I like AWS so much, and what separates them from other cloud offerings, is that they're doing stuff dev teams actually want and can use (not impenetrably complex tech we could use with some vendor consultancy). Import features like this are trivial for AWS to implement - but will be a step change for some customers.

Last generation vendors saying 'anyone can do what AWS are doing' is just crap. You can't buy the enthusiasm these guys have - and its not just one individual - if you talk to anyone from AWS (and they're often around nights/weekends to talk about stuff) they are in a different lane to the competition or vendors that should be the competition.

bravura• 17 years ago

Agreed, except I have found it pretty tough to figure out EC2. EC2 is fairly impenetrable at first.

kiwidrew• 17 years ago

Living in a country that's stuck on the end of an expensive and slow(ish) undersea cable for the past few years, sometimes my thoughts wander towards something like this. Even just a service (US-based) that would download a set of files (HTTP, FTP, bittorrent), burn them to DVD, and post them via airmail would be pretty handy. Bonus points if there was a way to use airline passengers as your couriers, such that the total latency might only be ~24 hours. :)

TJensen• 17 years ago

The airlines have services for that (see Delta Dash) without needing to use airline passengers. :)

nickb• 17 years ago

About 3 months ago, I needed >50Gb of data uploaded to AWS and I asked a friend of mine who works at Amazon for help. He asked around and told me to mail the external storage to Amazon with keys/bucket info and a special code. I UPSed the stuff and within 2 days of arrival, I received an email with a notification that the transfer was done. Later on, I got the storage back. Everything worked as advertised!

I guess there's so much demand for this sneakernet that they made an official service. I wonder how many other people asked for transfer help before today's announcement...

dfranke• 17 years ago

If it was only three months ago, more likely this service was already in development and you were customer #1 without knowing it.

3pt14159• 17 years ago

Unlikely. I have 2 contacts that did the same thing months ago. I actually thought that this was an already advertised service.

siculars• 17 years ago

hmm... maybe i'll sneakernet all my music/photos now. been waiting for a sneakernet uploading service. it was only a matter of time before they added one. wonder how long before smugmug adds a wraper around this and lets you send your dvd/smartcard/etc media and get it directly uploaded into your account.

callmeed• 17 years ago

This is going to be great for professional photographers (the industry I work in). I'm been advocating they use S3 for archiving for some time ... now it's easier to get your files there.

wmf• 17 years ago

Also check out Terascale Sneakernet from Jim Gray et al. from 2002: http://research.microsoft.com/apps/pubs/default.aspx?id=6457...

Bypass the internet for large transfers: AWS Import/Export

Related Stories