throwaway6845

I would really like you to clarify your intentions on serving non-HTML content.

I say this slightly nervously as a Cloudflare customer who serves some amount of binary data. One message is "it's ok if you're on a paid plan". Another is "it's not ok at any time". My suspicion is that "it's ok unless we notice you".

If you could come up with consistent understandable messaging that would help a lot. I don't mind paying (stay competitive against AWS and Hetzner and that's all I need) but the uncertainty is not good.

I'm sure it would become evident that they know more about spam and anti-spam efforts than anybody else in existence

Really?

I can point you to Hard Problems that have been solved better at little startups than at Google - or, indeed, at any other bigco. That's why acquisitions happen.

Why does Google having 1000 engineers working on a problem automatically mean they are the smartest?

there was also his Thaddeus Zu/Ashley Madison story, which was pure conjecture

It's a common HN fallacy that everyone here is a developer whose main life aim is to remain employable as a developer.

Lots of us are here to ship stuff; to provide viable products that customers will like.

Moving to AWS from Hetzner would not help me ship stuff. Moving to AWS from Hetzner would not result in a better product.

Mostly, headspace. If I run my own server, I just need to apply my existing Ubuntu sysadmin knowledge. If I use AWS, I have to learn a whole load of AWS-specific domain knowledge, starting with their utterly baffling product names. My time is more valuable than that.

Also, sheer cost. Literally everyone I know in my particular part of the industry uses Hetzner boxes. For what I do, it’s orders of magnitude cheaper than AWS.

This is very similar to my experience. I recently measured the difference on my site as 10%-15% (also a non-tech site, though sadly not at $8m a year yet!)

While I don't enjoy the blog either (partly for the reason you cite, and partly for the unspoken assumption behind many posts that FAANG-like companies are a good thing and a good working environment), I wouldn't phrase it like that. There is enough misogyny on the internet that starting a critique with "I hate" is not going to get your point across lucidly.

What's wrong with finding out how people use my site, so I can write more of the content they like, and make it easier for them to navigate through?

I agree. I have a big problem with UBO etc. blocking self-hosted, single-site Piwik. It feels like being penalised for doing the right thing.

Elon Musk (right) has drawn comparisons to Gerald Ratner in recent years

The 1980s British businessman who Musk most reminds me of is not Gerald Ratner but Robert Maxwell.

Other RDBMSs Are Clearly Less Egregious

Fortunately Google is consistent about enforcing encryption anywhere where passwords could be intercepted.

Oh, wait.

http://blog.elliottkember.com/chromes-insane-password-securi...

And if you disagree with them, you're "a novice".

https://news.ycombinator.com/item?id=6166886

Yeah, no way that would happen in Europe.

Oh wait.

http://www.telegraph.co.uk/technology/news/8438461/BT-and-Ph...

I don't want my ISP to inject JavaScript to random pages or analyze my traffic. That should be downright illegal. They should be like water supply company: provide me damn clean water and get out of my way.

I don't want my search engine to do that either. They should provide me damn accurate results and get out of my way.

Unfortunately, whereas I have a choice of several good ISPs here (UK), I have a choice of precisely one good search engine - Google - and it analyses my traffic to high heaven. (And I've tried DuckDuckGo, on several occasions for several weeks at a time, and I'm afraid it still sucks.)

OS X has always had it, albeit a fairly limited implementation. Try it in TextEdit.

A few years back I spoke at a conference about my product. I made some humorous, slightly disparaging comments about the Google-owned competitor.

Two days later my site disappeared from Google SERPs with no ostensible reason. I later found out there were Googlers in the audience for my talk.

Correlation is not causation, so no proof, just suspicion.

It is _already_ behind a sign-up with a checkbox. They scraped their way past that too.

We're not talking generic "bots".

We're talking a custom scraper written for this site and this site only.

Yes, I am expecting the people who spend hours inspecting the source of my site, and then writing a custom scraper for it, to spend 30 seconds reading the T&Cs first.

The analogy was "doing something anyway because you might be told 'no'", not "my server behaves like a canine quadruped". Copyright infringement is also potentially criminal in the UK, it's not as simple as you suggest.

But whatever. It just saddens me that the internet is a constant "don't be a dick" battle with companies like the scraper guys.

(edit - understood :) )

Well, yeah, like if you ask someone to sell you their dog and they say "no". Doesn't justify stealing the dog.

In what world is "engage a third-party scraping company" a better option than "drop a quick email to the site operator"?

Yes. Did that after this episode.

I built a simple CRUD app for a previous (small) employer. Nothing special technology-wise, but a good concept, sound business model, and backed up with a couple of full-time staff creating content for it. Line one of the T&Cs was "no scraping". Business model was based on sales to individual users but we were prepared to do analysis in aggregate if asked.

A scraper company, funded by magic money (Knight Foundation grants) and $1m of VC, convinced a (UK) Government department to pay them to scrape our site for some analysis the department wanted. They'd never contacted us, never asked for permission, never asked if we could supply the data. Our company was bumping along at this point and having to lay people off. Income from a nice lucrative Government contract would have kept a couple more people in work.

The scraper company's FAQ was, in my view, full-on unethical:

"we check the robots.txt file. If the site permits robots in general to scrape their site (NOT just GoogleBot!), then we will do so. We will make no effort to look for other terms and conditions as well."

You will ostentatiously "make no effort to look" for T&Cs in case they prohibit the significant contract you're about to sign with the Government? Whoa.

So how I feel about web scraping is simple: "don't be evil". If you're diverting income or traffic from the original site, don't do it. If you're genuinely adding value, go for it, but be open, be prepared to work with the original site, and be prepared to accede to their wishes.

Ok, I'll take your word for it, but that said HMRC is the only way in which I ever interact with the Government online.

If GDS can't get their claws into HMRC then Government digital (lower case) is pretty broken.

It's not a message that needs to be delivered securely. It is literally just "you need to fill in your tax return by the same date everyone else in Britain needs to fill in their tax return". That could have been included in the email body with no security implications.

It's basically crying wolf. Next time they have something really important to tell me, I suspect I'll just go "nah, it wasn't important last time, I can't be arsed to spend three minutes logging in" and delete it.

This is pretty horrifying.

But almost as bad: websites that insist on over-elaborate security measures for trivial stuff. Take a bow, HM Revenue & Customs:

You’ve got a new message from HMRC

Dear Fred

You have a new message from HMRC about Self Assessment.

To view it, sign in to your HMRC online account. For security reasons, we have not included a link with this email.

Why you got this email

You chose to get paperless notifications instead of letters by post. This means we send you an email to let you know you have a new message in your account.

From HMRC Self Assessment

And HMRC have mandatory 2FA. So to read the spam they've sent me - and it is pretty much spam, it says "you need to do your self-assessment before next January", I know that already - I need to go through the rigmarole of entering my Government Gateway number, which I don't remember but starts with a 4 or something and hopefully that will be enough for Chrome to autofill it, then authing with my mobile phone. Which I think I left upstairs or something. Wait while I ring it with the landline to find where it is.

Seriously, I might just go back to getting letters by post.

Edit: No. My Government Gateway number which starts with a 4 is my company one. My Self-Assessment login appears to be a different number.

People elsewhere in the world, whenever anyone tells you that the UK Government Digital Service is a beacon of usability and good practice, please don't believe them.

...ok, I goofed. Getting confused between Inc and Mic (three letters, two of which are I and C, very similar logos). Apologies, please ignore above comment.

Inc Magazine is very identity politics, very much written by privileged Silicon Valley kids with preconceived notions of diversity. I gave up reading it a long while ago and I'm as liberal-left as they come.

Countdown until a JS extension that takes a normal <input> field and uses &bull; characters to make it look like a password field without tripping Chrome's detector...

I don't see this as substantially different from the ads that appear on google.com telling me to switch to Chrome when I have the temerity to access it from Safari.