Online streaming sites and URL obfuscation

https://thesquareplanet.com/blog/resistance-is-futile

by Jonhoo • 11 years ago

67 16 11 years ago

hackerboos• 11 years ago

rtmpdump [1] and livestreamer [2] are other good tools for this type of thing.

I've always had trouble trying to rip JWPlayer videos that sit behind a login. I did manage it but I had to stream (second-by-second) the entire video.

[1] https://rtmpdump.mplayerhq.hu/

[2] http://docs.livestreamer.io/

Pengwin• 11 years ago

Youtube-dl [1] is my go to tool for pulling video content out of a page, and its source [2] is a nice lesson in de-obfuscating the sites it supports

[1] = https://rg3.github.io/youtube-dl/ [2] - https://github.com/rg3/youtube-dl/tree/master/youtube_dl/ext...

tlow• 11 years ago

For a long time I've used the by hand method of URL extraction either inspecting source code and searching for .flv or .mp4 filetypes (or simply "file:*"). The other method I use is the built in developer tools "network" tab of chrome under "inspect element". So far, I've only encountered one host (sorry I cannot remember the name) which I wasn't able to extract the source file via one of these two methods.

Jonhoo4• 11 years ago

Yes, the second method I used to use quite frequently, but it means you have to open your browser, click play to get the video to load (which will spawn a torrent of ad pop-ups and captcha prompts and whatnot). This way I can get the download straight from the command line.

NickHaflinger• 11 years ago

How do you use streamsh?

https://github.com/jonhoo/streamsh/blob/master/tricks/jwplay...

Jonhoo4• 11 years ago

stream.sh <video URL (preferably the embed URL)> <filename to download to (without extension)>

tobltobs• 11 years ago

At the end he says something like "We need Encrypted Media Extensions to get ad free streaming...". Hmm, never thought about that this way. Maybe he is right, but couldn't it be that with EME the ads will not be sitting next to the player, instead the streamed video will contain the ads? Or does anybody believe (enough) people will pay for streaming on youtube or similar pages?

madeofpalk• 11 years ago

the ads will not be sitting next to the player, instead the streamed video will contain the ads?

This tech already exists, and it's called 'Server Side Ad Insertion', with Brightcove, The Platform, Adobe and Google all offering solutions.

It's actually pretty cool - it works by instead of streaming a baked file, the client gets a 'live stream' URL and ads are inserted into that stream, dynamically, just like regular video. It's like having `badge.svg` image that's actually a PHP script that dynamically creates an SVG.

I'm not overly familiar with video tech, so I've definitely butchered the exclamation.

TheLoneWolfling• 11 years ago

And then ad blockers will snip out the ad portion of the stream and the cat and mouse game will continue.

anseljh• 11 years ago

"streaming freedom" is a really clever euphemism.

stabilo2• 11 years ago

Is a shorter solution possible?

Is decompiling SWF and/or de-obfuscating Javascript really necessary?

Years ago, I approached these sites like the author in the blog post. I spent hours reading about Flash and figuring out the schemes websites used. I decompiled swf. I also tried rtmpdump, livestreamer and other all-in-one solutions. I used early youtube downloaders (clive, etc) and later quvi. I also used /dev/bpf to capture HTTP requests. I tried it all.

I can relate to what the author is trying to do. I never settle for "streaming". Download only.

However I found over the years either the websites have made things easier or I was simply trying too hard. Or maybe I just am not interested in the type of video that is served in this way. For some reason it takes much less effort now.

To make sure I am not imagining this, I decided to try one of the author's examples.

I chose projectfreetv.so

1. I followed a link on the main page for some TV show I have never seen.

2. I read the HTML.

3. I noticed /watch/?aff_id= URL's.

4. I chose the first one, 493165, and followed it.

5. I read the HTML.

6. I noticed a URL pointing to a website that serves TV shows and followed it.

7. I read the HTML.

8. I noticed a URL pointing to an MP4 file.

9. I downloaded the file.

I assume I got the TV episode on offer. The screen size is a little small but the file was about 89MB and appeared to be the full episode.

Total time: less than 10min

What did I use? sed, netcat, less and ftp for the download. The entire process is a one-liner.

If anyone has some more examples to try, please list them. I would welcome the challenge.

Jonhoo4• 11 years ago

Author here: for some sites, that approach works well, and this is what I originally did (and why I decided to write streamsh in bash). Have a look in the "tricks" directory and you'll see some of the much simpler solutions that are tried first. However, an increasing number of sites try to hide the URL, and through increasingly complicated means. The article is about vidbull.com, but there are others. They often also layer multiple obfuscators (vidbull uses both JS obfuscation and URL encryption for example). While it's true you can get pretty far just by scanning for mp4s on some mirrors, they are becoming fewer and farther between.

stabilo2• 11 years ago

vidbull looks more like the old "file storage locker" sites that were popular before the megaupload crackdown. Isn't this the same content you can find using torrent sites? Is it likely that vidbull would have something for which no torrent would ever exist? Sorry for my ignorance.

wodenokoto• 11 years ago

Many of these sites have started supporting iOS. Wouldn't it be easier to spoof iOS Safari and then only have to dig around in JavaScript?

Online streaming sites and URL obfuscation

Related Stories