Unlock Articles with Paywallskip

https://www.paywallskip.com/

by francocanzani • 2 years ago

55 62 2 years ago

Hi! Our project addresses the limitations of existing paywall bypass tools by implementing a dynamic, community-driven approach. Key features include:

Real-time Adaptive Blacklist:

Constantly updated database of paywalled sites and effective bypass methods User-driven reporting system for quick adaptation to paywall changes Significantly faster response to new paywalls compared to static solutions

Multi-Method Bypass Arsenal:

Unlike single-method solutions (e.g., 12ft.io's cache access), we employ various techniques Methods include: User-Agent spoofing, Referer header manipulation, JS disabling post-load, and web archive fallbacks (Wayback Machine, archive.is, Google cache) Our blacklist determines the most effective method per site, improving success rates

Site-Specific Solutions:

Tracking individual websites allows for custom bypass methods when general approaches fail Parsed and validated HTML output ensures content integrity

We believe this approach offers a more robust and adaptable solution to paywall bypassing. We're eager to hear the community's thoughts and potential improvements.

Ancapistani3• 2 years ago

Is this open or shared source?

Need help?

francocanzani8• 2 years ago

I'm really doubtful on this because most of these open source apps get banned from Github. Probably will make a dummy account to open issues and always have a copy local and in Gitlab. Should be able to clean the code a bit and share it soon.

foobarqux2• 2 years ago

Doesn’t seem to work on wsj, which is the only site archive.is doesn’t work on consistently anymore. So why not just use archive.is?

francocanzani8• 2 years ago

Will have a look. It worked earlier today.

ventegus• 2 years ago

wsj has easy-to-bypass paywall but hard-to-bypass antibot/antiscraping solution, so the browser extensions work better than hosted bypassers

holistio8• 2 years ago

This is going to be a shameless plug here.

I'm also frustrated by how paywalls work, having to endlessly subscribe to yet another newspaper or newsletter, just because you want to read one article.

But this approach is... stealing. When it comes to big media like the NYT, it might not feel like it is, but at the end of the day, it's people doing the background research, writing the content, editing, etc.

This is _not_ the solution.

Our alternative in the making offers a legal, fair alternative: buy and sell content by the piece.

It's available at https://vernus.one

If you have a blog or newsletter, or if you're a journalist, I'm happy to help in setting up your publication and start earning in a transparent manner.

m000• 2 years ago

Can I get my money back if it turns out that some NYT article was outright lies, probably already paid for by some undisclosed shady entity?

holistio8• 2 years ago

I wish you could. I'd like Vernus to champion indie journalism, mainstream media is dying before our eyes anyway.

napoleoncomplex2• 2 years ago

Do you get your money back if you go to the store and buy some new food/fruit/snack you don't like the taste of?

No, you throw it away, and probably won't buy it again. If you don't like NYT, don't buy from them.

If NYT is like an avocado for you, sometimes ripe and delicious, sometimes unripe, sometimes rotten, you get to decide how often you're gonna buy avocados, or if you'll develop your own methods of avocado testing before buying to increase your odds. In no case do you get to take the avocado skin back to the store asking for a refund.

rrr_oh_man2• 2 years ago

Too salesy

holistio8• 2 years ago

My comment or the site? If the former, I acknowledge that, I got a bit triggered by YACTS (yet another content theft service). If the latter, I'm genuinely curious and happy to make changes to our copy.

Paywallskip seems to stealsy to me FWIW.

Ancapistani3• 2 years ago

Disputing the space and then resolving the mess is a tried and true tactic in this industry.

Just because this site isn’t directly paying the content providers now doesn’t mean they won’t in the future. Providing it free, even with ambiguous legality, can allow them to build enough of a following to give them leverage in future negotiations.

holistio8• 2 years ago

It's not directly not paying, it's not paying at all.

I don't see how this is different from disputing the banking sector by conducting a heist.

rendall• 2 years ago

Does paywallskip scrape archive.is and archive.org?

francocanzani8• 2 years ago

It fallbacks to archives, yes. Basically I use different User-Agent headers, different Referer headers, it tries disabling javascript once the page has loaded and the fallback is to fetch from web archives (Wayback Machine, archive.is, Google cache).

Then the HTML is validated and parsed.

rrr_oh_man2• 2 years ago

What will you do when the lawyers come for you?

francocanzani8• 2 years ago

The legal landscape surrounding this issue remains ambiguous. I've documented my analysis in the legal section of my website. Typically, the consequence is domain takedowns, which is why I proactively purchased 10 domains as a precautionary measure.

https://www.paywallskip.com/posts/legal

hiergiltdiestfu• 2 years ago

aaomidi• 2 years ago

Get a .is domain potentially. They kinda don’t really care about these takedown requests IIRC.

frankacter• 2 years ago

Are you considering a chrome extension to automate the process from the client perspective?

francocanzani8• 2 years ago

Yes, is being developed and will launch this week. Just click and go, nothing fancy.

DataDaemon• 2 years ago

not every hero wears a cape

upwardbound• 2 years ago

"but all villains do"

(This quip means that if a demagogue aggrandizes themself as a hero or savior, you should be suspicious of whether they really are as saintly as they claim to be.)

Edit:

I'm not saying this quip about paywallskip, which I think is a kindhearted service because it helps reduce the information gap, or digital divide, between the middle class and those stuck in poverty.

Rather, the joke/warning is about demagogues in politics or in wealthy churches, stuff like that.

Not about this post.

Paywallskip never called themselves a hero - an appreciative user did, which is authentic and sweet.

I think no one should ever call themselves a hero.

Just help people, like Paywallskip is doing.

I guess that what real heroes always say, both in movies and in real life, is "I was just doing my job". See e.g. this school bus driver who saved a child's life:

https://www.youtube.com/shorts/xnW3NHbMTRs

(short, inspiring video)

emmet• 2 years ago

Not a fan of paywalls, but the framing of the pricing comparison is just so funny to me.

Like the guy who comes into the pub offering to sell dubiously acquired meat for a bargain price.

brtkdotse• 2 years ago

I’d like something like this, but instead of bypassing the paywalls and stealing the articles it’d pay a few cents for each article and charge me at the end of the month.

I really, really want to pay but I really, really don’t want to set up a subscription to read a one-off article.

bluescrn• 2 years ago

Nobody's every been prepared to try microtransactions that are actually micro.

I'd like a browser addon that warns of paywalled links before I click them, perhaps adds a padlock icon or dollar sign (maybe with required subscription/cost on hover), to avoid nonsense like 'the first paragraph or two is free, then you hit the paywall'.

Even better if it can do the same for region-locked sites and adblocker-blocking sites, or if on mobile, sites that nag/force you to use apps instead of the web...

nelox• 2 years ago

Blendle used to provide it, but alas, no more:

https://en.m.wikipedia.org/wiki/Blendle

salviati2• 2 years ago

I thought the Brave browser would provide such a solution. I was very disappointed to find out it's not an intended use case.

yard2010• 2 years ago

You are not stealing anything, unless you find a way to delete the source article after you downloaded it. It might break copyright rules. I wouldn't recommend doing either though.

lynx232• 2 years ago

Hmm, I think the victims of organized corporate espionage might see this completely different. Its totally enough if "agents" of a certain country known for not really caring about copyright copy the plans for your new product. They dont have to delete yoour original plans to do damage.

alex_hitchins• 2 years ago

I'm sure there was a startup a few years ago trying to achieve this. I can't recall the name sadly, it was 10+ years ago. Agar or something perhaps?

I believe you topped up a balance and if the site supported it could access per-article resouces for a one off fee. For this to really work it needs to be site independant however.

truculent• 2 years ago

“Is using PaywallSkip legal?” is in the FAQs, but not “Is using PaywallSkip ethical?”

Why do we feel so entitled to other people’s work as soon as it’s published on the internet?

When Homer Simpson reads magazines at the convenience store without paying, we understand that he’s behaving poorly. Why do we feel differently now?

austinjp2• 2 years ago

Because the corner store has, in many cases, become a gigantic multinational monster that exploits the authors and the readers.

jampekka• 2 years ago

Why do we feel so entitled to other people’s work as soon as it’s published on the internet?

Because intellectual property is a broken and harmful system.

mathgeek• 2 years ago

When Homer Simpson reads magazines at the convenience store without paying

A more specific example would be ripping off the plastic packaging to read a magazine. The ones without extra packaging are designed and for a long time expected to be browsed.

salviati2• 2 years ago

I tried to subscribe to an online newspaper in my country.

Subscription was very easy. When I wanted to end it, they made it purposefully hard for me to do. I think it's unacceptable, and should be outlawed: the process to unsubscribe should be as hard (or easy) as the one to subscribe.

Since newspapers (at least that specific one) have no problem doing this as long as it's legal, why should the public be more observant of ethics?

freefaler• 2 years ago

If Revolut is available in your country, create a new virtual card for each subscription and when you can't/don't want to pay them anymore just cancel the card in Revolut.

lynx232• 2 years ago

Its the old leftist anarchism attitude, stealing from someone bigger then you is no crime. Why? Because the system is evil anyways, so no need to play nice. Its starts with people grabbing a newspaper from the stand, and not throwing in the coins. And from there, its uphill... Some people think its totally fine to install a specific app to bypass youtube ads, instead of buying premium. 30 years ago, these people were cracking pay tv with their computers. And it was cool!

P.S.: I am only reporting, this is not my attitude.

BriggyDwiggs42• 2 years ago

some people think it’s okay to install a specific app to bypass youtube ads

My god man with the way those things have been trending it makes you a saint lol

OutOfHere• 2 years ago

stealing from someone bigger [than] you is no crime.

Although it's finely said, in a related worldview, I would adapt it to say:

stealing from a publicly traded corporation is not a crime because they're not a person anyway

This takes away the pain inflicted on someone when stealing from that individual. Traded corporations carry no pain.

nkrisc• 2 years ago

Do you also support skipping the paywall for services like AWS as well? That would be useful.

francocanzani8• 2 years ago

That would take the fun out of AWS

master_crab• 2 years ago

Seems to be broken. Get a lot of server side error messages

Unlock Articles with Paywallskip

Related Stories