Justin.tv ending archiving, deleting all archived videos after June 14

https://help.justin.tv/entries/41803380-Changes-to-Video-Archive-System

by makomk • 12 years ago

149 74 12 years ago

help.justin.tv

protomyth• 12 years ago

A little over a week doesn't seem like enough time for the news to get out and people to download all their content.

deftnerd• 12 years ago

I'm not a Justin.tv user, but it frustrates me when organizations make big decisions like this without much time for their users to react. I can only assume that they're bleeding money and they're trying to quickly cut their operating costs.

Someone should contact the Archive.org and see if they can get copies of all the public archives of these videos.

tim333• 12 years ago

Yeah, 8 days notice isn't much though at least it's better than Yahoo's shutting down Everyblock with no notice at all. I see Everyblock is back under new ownership but I'm not sure they've got the old data.

slig• 12 years ago

IIRC, it's NBC who shut Everyblock down.

taurath• 12 years ago

Justin.tv owns Twitch, which is recently rumored to be in acquisition talks with Youtube for $1bil. I don't think they're bleeding money at this point.

kevinflo• 12 years ago

Bleeding money and being in acquisition talks are not mutually exclusive.

lost-theory• 12 years ago

Perhaps they want to make this change before the acquisition to spare Google the bad press of "shutting down yet another service".

bruceb2• 12 years ago

So only 1 week notice? Bit harsh. Seems 30 days would be more standard when shutting a service or feature.

pgrote• 12 years ago

It could be a cost elimination.

niels_olson• 12 years ago

Could be? I'm amazed they were keeping all that video this long.

bruceb2• 12 years ago

I am sure it is and I can understand eliminating this feature but why not give users more warning?

acdha• 12 years ago

Likely - also, if true, basically the equivalent of putting a banner on their site saying “We don't value our users!”. Even a minimally competent management team should be able to look at the burn rate and make decisions more than a few days out.

pavel_lishin• 12 years ago

I wonder what it would take to transload them all to youtube.

loceng• 12 years ago

Perhaps more than a week?

ja2ke• 12 years ago

Twitch.tv has offered this service for ages, but maybe it started offering it after splitting from the Justin codebase?

im3w1l• 12 years ago

We found that more than half of our VODs are unwatched (with 0 or 1 total views), while the vast majority are rarely watched (with 10 or less views).

Isn't this the standard for content? Anyone know similar statistics for image hoster, url minifiers, youtube, appstore etc?

sp332• 12 years ago

It seems like they asked the wrong question. It's not about how many people want to watch archived video, it's about how many people want to make an archive of a video. That said, I wonder if the amount of money they save is enough to offset the users they lose to other services.

jonny_eh• 12 years ago

I think the question they asked was "how much does it cost to host all this video?"

Then for their post, they answered the question they wished they were asked, like a politician does in an interview.

Aqueous• 12 years ago

Perhaps they should just keep the archive videos for a week or a couple of days after the broadcast. That gives a window for users to download archived copies and reduces the amount of storage they need on their end. It also means that users aren't forced to generate video archives themselves, and can still rely on the service for this function. Maybe for videos that accrue a certain amount of unique views during the grace period, keep them around in a sort of "Classics" library?

ggreenbe2• 12 years ago

I like the idea of something like 24hour retention. If you just missed a broadcast by say an hour or live in another timezone +12hours then maybe watching it later within the 24 hour grace period would be good.

kcbanner• 12 years ago

Twitch.tv uses a similar system

maccard• 12 years ago

Twitch is owned by Justin.tv

pronoiac• 12 years ago

For comparison, Ustream announced a few months ago that they'd only retain archived video for a month for free accounts.

Asparagirl3• 12 years ago

If the destruction of all this unique user-generated content pisses you off -- and it should -- then feel free to join ArchiveTeam and help save this (and other) content for the Internet Archive. Check out the ArchiveTeam wiki at archiveteam.org, or come on over to #archiveteam and #archiveteam-bs on efnet throughout this week to watch a self-organizing data rescue mission happen in real time. (There will probably be a new dedicated channel spun off for this project later today or tomorrow.)

In the meantime, programs like youtube-dl and livestreamer are your friends.

http://rg3.github.io/youtube-dl/documentation.html http://livestreamer.readthedocs.org/en/latest/

IvyMike2• 12 years ago

To elaborate, the ArchiveTeam distributes a very simple-to-use virtual machine image that you can run. When you do so, your machine will help download and collect from disappearing websites. I don't think they've got scripts for justin.tv yet, but it sounds like they are going to, so now would be a good time to get your VM ready.

It's easy, and the more people who run the machine, the more power the ArchiveTeam has at their disposal. I encourage everyone to give it a look.

You can get the virtual machine here: http://tracker.archiveteam.org/

Asparagirl3• 12 years ago

Right, the idea is that justin.tv could be formatted as a new project for the ArchiveTeam Warrior -- which you can run on your laptop or a VM or a cloud server or a few Amazon instances, or whatever. But with only a week's notice before the shutdown, I don't know whether they can get the customized scripts up and running in time. They're working on extracting a list of user and video id's right now.

ddorian43• 12 years ago

it has stopped working for me, what about you, it still works ?

MichaelStubbs• 12 years ago

It's worth noting that you don't have to use a virtual machine. If you're familiar with compiling basic utilities and installing Python packages, then you can run it without the overhead of a virtual machine. It's generally all on the wiki/Github.

g_lined• 12 years ago

I downloaded the VM today and spent a fair while trying to diagnose a CheckIP error. It turns out, after going on the IRC channel that none of the projects on the VM are up to date and hence the VM can't be used to help the project. Hence the error. If I see the website update to mention justin.tv I'll certainly fire it up. But it seems a shame that there is nothing I can do to help with their project as a whole; I couldn't even see any way to donate - I guess I should just donate to archive.org.

Asparagirl3• 12 years ago

As expected, the dedicated channel for the data rescue has now been set up: join #justouttv on efnet. Right now, coders with Python, Lua, and/or Redis knowledge would probably be most in demand, given the existing toolset. Hopefully very soon there will be a new Warrior project formally set up, and then anyone who can run a VM or an AMI or install the tools from the command line would be able to join in to distribute the downloading.

During the Hyves.nl rescue in late 2013, ArchiveTeam did accept donations to cover Amazon instances earmarked for that project, so it's possible they may do something like that again this time.

In case people are curious, the Warrior code and seesaw code are here:

https://github.com/archiveteam/warrior-code2 https://github.com/ArchiveTeam/seesaw-kit

Smerity• 12 years ago

It honestly disturbs me that Myspace, Justin.tv, and these other companies don't seem to ever consider shipping their soon-to-be-deceased data to the Internet Archive. Why not help preservation a little? It'd likely be simpler to handle than the oncoming denial of preservation defense that the ArchiveTeam launches. Have there been any examples of companies doing this?

You might think that this is all useless information, but I guarantee we'll look back in likely only a few years and think in horror at all that we've destroyed.

The best art blog about a half decade including a kid who's dead now just disappeared. When Myspace deleted all of their content, I lost all the content of my dead high school friend (turns out you need to back up everyone and thing you care about, not just yourself[1]) and @maxfenton mentioned "The best art blog about a half decade including a kid who's dead now just disappeared.". It's not an isolated problem.

I almost feel like we need a name-and-shame registry and/or public awareness campaign. I'd love to see this written into the user rights of a website when it launches.

"If the website is to be shut down, all publicly accessible content will be sent to the Internet Archive. If you would like to opt out, tick this box."

[1] http://smerity.com/articles/2013/deleted_digital_tombstones....

mantrax53• 12 years ago

"If the destruction of all this unique user-generated content pisses you off -- and it should"

I'm not sure why it should. When did we get this strange idea that when you make some piece of content, no matter if it's quality or not, it deserves to pollute the cyberspace forever.

I realize this is an unpopular opinion, but one of the reasons nature "works" is that everything old is allowed to dissolve and become the base ground for the new that will come after it.

People naturally copy content that they find interesting. And content which isn't that interesting naturally gets lost. Maybe that lost content stuck around enough to inspire someone to do something in that spirit later or (which also counts), but it doesn't survive itself.

Imagine if everyone gets Glasses one day and starts doing non-stop 1080p video stream of their lives. Are we going to be "pissed off" at that being deleted too?

If you do the stats you'll notice that over 99.99% of the content in archive.org is never accessed. Nobody cares.

And that's how it should be. We should really allow content to die. I have the firm belief that information doesn't just "want" to be free. It also wants to be allowed to die gracefully at some point.

ivank• 12 years ago

People sometimes make copies of things they find interesting, but those copies are rarely accessible to the public. It can be pretty hard to get your hands on something after it disappears from the web. For example, forums.truecrypt.org went down three days ago, it's of interest to some people, and you're going to have a hell of a time finding some guy who happened to launch HTTrack on it before it went down.

voltagex_• 12 years ago

I'm currently running through a Peter Norton assembly tutorial from the 1980s. It's already helped me understand the code better than I have before. Should that content have died, just because it's old?

sheltgor• 12 years ago

Who's to say what we'll find important down the line? Quite often what seems mundane and not worth the effort to copy or maintain turns out, in hindsight, to have true value. One of if not the most important insights in studying previous cultures comes from going through their trash heaps. Similarly, while they could seem pointless, a lot of this content could provide unique insight on our own culture down the line that 'valuable' pieces saved and curated can't.

CocaKoala• 12 years ago

There's value in preserving the mundane. If we only preserve things which are noteworthy, the cultural artifacts, the special occasions, the celebrations and wedding dresses, it's easy to lose track of the fact that it's worth remembering what everyday life was like, too.

dredmorbius2• 12 years ago

Two week notice for storage policy changes smells like financial concerns to me.

serf• 12 years ago

well, to be fair, nearly any storage policy change will smell that way.

dredmorbius2• 12 years ago

It's one thing to say "we're phasing this out over 12 months" (or even, say, six).

Two weeks is freaking desperation.

aleem• 12 years ago

This is reminiscent of Google's (and others, to be fair) build-first, monetise-later approach to products. Features get cut, entire products get slashed and users end up losing their data and time invested.

At least Google had acted partly responsibly by providing ample notice and allowing users to download their data before shutting down Google Reader.

In the case of Justin.tv, I would propose they at least take a measured approach and start by deleting videos that haven't received a single view in over a 6 months then work their way from there.

rhizome• 12 years ago

In 2007, this was THE FUTURE, where UGC was going to be the democratizing content tide that would lift all of our boats. Turns out it all devolved into ad platforms anyway, just like 1997.

dylanrw• 12 years ago

I like how they phrase this like it's a considerate product decision, while it's essentially them not wanting to shoulder the expense of storage. Without knowing more about their product internals an immediately obvious and important feature getting the axe from this is highlights. Example: I will stream and then go back the next day and cut the video down and take the snippets and post them to youtube. These highlights then lead to interest in the channel which leads to more viewers. Now I need to archive all my stuff locally, and upload it over my mediocre connection. If they limited the archival time window to 4 hours after the end of the broadcast they would have their drastic reduction in infrastructure cost while not totally removing a valid and arguably important feature.

kevingadd• 12 years ago

I think that's how Twitch works by default now. They archive the whole stream for like a week so you can Highlight it, and then it's auto-deleted (unless you've configured it to keep everything).

kevinwang• 12 years ago

I believe the default now is that nothing is saved, not even for a week. However, each user can configure a setting that does cause Twitch to save the stream for about a week, before it's deleted.

In order to fully archive a stream "permanently", a user needs to view each archived recording individually and select an option from that page to save it forever.

Yver2• 12 years ago

The page currently says "All VODs will be removed after June 8, 2014. We recommend downloading your recorded videos before the date."

makomk• 12 years ago

Yeah, I'm not sure why the title got changed to say June 14th.

Yver2• 12 years ago

Oh, that's even weirder. I thought the page changed after you posted the link, not the other way around.

instakill• 12 years ago

That really isn't a lot of notice.

jacquesm• 12 years ago

If they're cutting things this fine I fear for the future of justin.tv, this does not sound like a well thought out corporate move, it sounds like panic. Imagine coming back from your holiday in two weeks time and finding all your stuff wiped out. Ridiculous.

dtech• 12 years ago

I know Justin.tv and Twitch.tv are related, does anyone know if this policy will also reflect twitch?

abstractbill• 12 years ago

The policy will only affect Justin.tv, not Twitch.

swang• 12 years ago

Of course this is financial.

Justin.tv is not a profit center for Twitch Interactive, so they need to cut costs. If 50% of VODs are viewed <= 1 time, and most viewed <=10 times, why not just leave them there? Obviously storing them costs money, but there probably isn't a lot of bandwidth costs associated with it.

I don't have anything against what they're doing as much as the sugarcoating of "focusing on live video"

This also lends more credence that they're getting bought out soon and want to look good on the cost sheets.

SoftwareMaven• 12 years ago

and want to look good on cost sheets

Ding, ding, ding. We have a winner!

Seriously, there can be no explanation for the ridiculously short notice other than YouTube said, "You look good, but your costs are just too high." (Or, if not YouTube, somebody else Twitch is trying to get into a bidding war with YouTube.)

Any other reason and management would have looked at the costs and said, "We need to transition out to save money," and provided a reasonable runway for their customers. As it is, this has nothing to do with making things better for their customers.

ghaff• 12 years ago

On the one hand, I have a fair bit of sympathy for the preservationist school of though--and the Internet Archive does great work. I also consider it very fortunate that much (though not all) of Usenet was preserved when it may well not have been.

OTOH:

We found that more than half of our VODs are unwatched (with 0 or 1 total views), while the vast majority are rarely watched (with 10 or less views).

The reality is that there is an increasingly indescribably volume of user-created "stuff" out there and it's pretty impractical to preserve all of it. And when it's not on a well-known site that's shutting down (think Geocities) it mostly sinks beneath the waves without anyone really noticing. I could probably name any number of online magazines/sites which went away or restructured and whose content is no longer available. I'm not saying that's a good thing but it's hard for me to get too worked up in most of these cases.

chroma• 12 years ago

A few Defcons ago, Jason Scott gave an excellent talk about this failure mode[1]. Basically, companies host user-generated content for free. Then years later, they destroy it with little or even no notice. Sturgeon's law applies; most of the deleted content is crap. But a lot of it matters to the people who made it. More importantly, a small fraction of it is stuff that future historians would kill for. Collectively, companies that engage in this behavior are burning historical evidence.

1. Archive Team: A Distributed Preservation of Service Attack http://www.youtube.com/watch?v=-2ZTmuX3cog

maxerickson• 12 years ago

I find it hard to get too wound up about an only copy that has been turned over to a third party. I get that the user understanding the situation well can be a problem, but it isn't a very careful thing to do.

Siecje• 12 years ago

They should only keep content for a week if there is no views.

Having an additional recording is convent, now people have to record locally at the same time as streaming.

blat001• 12 years ago

I wonder if it is relate to copyright issues associated with the sale of twitch.tv

As in Google saying they can't deal with these videos without proper acceptance of their terms of service.

The timing seems to indicate they are trying to offload the content ASAP.

From a cost perspective, they are already paying for it and 1 week vs 1 month would not make that much of a difference if it was a "product feature"

dobbsbob• 12 years ago

i only use JTV to nerd out in the trek streaming channels and make fun of rikers lack of beard in chat during season 1 TNG

Paul123455342• 12 years ago

I understand people need to make business decisions but short notice on something like this isn't exactly a classy move.

AliAdams• 12 years ago

I don't see why the decision is so digital. Surely a 1 week hold coupled with the ability to move the video to YouTube (or simply the latter as an option at the end of a live stream) would take away the huge hosting costs but give content creators the chance to save certain videos if they wanted to.

brianbreslin• 12 years ago

Its interesting if you read their FAQ, they are eliminating their premium service which in theory should have included archiving. My first thought was why not charge people for permanent archiving? Guess not enough people are paying to even justify the consumer premium model.

I feel like justin.tv's peak was 5+ years ago, twitch seems to have been their real business in the last 3 years.

Justin.tv ending archiving, deleting all archived videos after June 14

Related Stories