imilk

Well a good step is to never give your clients full admin permissions in WP so that they can install plugins or mess with theme files.

Well we have hosted 10 small business WP sites per $10 DigitalOcean droplet for the last decade. There are not additional plugin costs on any of them. And there has been no real maintenance needed.

I'm not saying WP is great. Taking over a WP project from someone else can be daunting in tech debt and weird choices. But in terms of having a simple brochure website for businesses that get < 10k weekly visitors, it's pretty quick, cheap, and easy.

let them SFTP some markdown files

Have you ever worked with any SMBs before? This is at least 5 technical levels above their head. Would make as much sense as telling them, "just use this CLI tool".

We're talking about people who will email you from their phone that the website is down, but it turns out it's just their home internet that is down.

Or think that the website disappeared from the internet. When in reality it's now the #2 result in google and they never knew they you could type a URL directly into the browser.

Exactly, since they can bill those as conversions (like phone calls, bookings, reservations, etc) rather than simply website visits

that's why systems architects are supposed to get paid well, because there are a lot of different stakeholders to consider

Not really because most of the links in current AI overviews for many queries keep you in the Google ecosystem, such as links to Google Business profiles on Google Maps. And Google has recently changed Google Business profiles to make outbound website links less prominent compared to the paid features on that platform.

Most people here have commented that they prefer AI responses to raw search results

Strangely enough, I've seen the exact opposite response on here. Especially since the AI overviews are often plain wrong and/or misleading. Many others like myself also prefer to get information directly from 1st parties, rather than whatever sausage has been produced through the black box information meat grinder we call AI.

A systems architecture perspective should be very very inclusive of the business perspective.

Have used supabase a bunch over the last few years, but between this and open auth issues that haven't been fix for over a year [0], I'm starting to get a little wary on trusting them with sensitive data/applications.

[0] https://github.com/supabase/auth-js/issues/888

We never distinguished automations from people though, that makes no sense on the internet.

LOL I see you've never sold anything on the internet, ran a website that is supposed to generate leads, or had to gauge the effectiveness of an ad campaign. There is a huge part of the internet that relies on real humans doing things on websites. And ignoring that is insane.

Google has clearly decided to keep users on their platform longer, hoping that this will lead to more ad clicks. There is a clear reason why AI overviews very seldomly link to outside websites, and why website links are much more hidden on Google Maps/Business Profiles. More time spent on the Google platform means more likely that someone will eventually click an ad.

Also - I noticed a pretty huge outcry when AI overviews were introduced to search. Can you show me all the people who enjoy the experience of using them more than not?

Probably because that's one of the only things you can say to Google these days that will escalate it from an low level support agent on the other side of the world.

Vastly varies on what type of website you have, how many pages you have, and how often they are updated. We routinely see 1000's of requests per minute coming from AI bots and the scraping lasts for hours. Enough to make up 20-30% of overall requests to the server.

...that's literally the entire point of this article. People don't want their websites being de-listed from the monopoly that controls organic traffic. At the same time they would like some control over stopping companies (in this case, the same company that controls the organic search monopoly) from scraping and repurposing their content so their the traffic to their website doesn't decrease.

Why is it such an issue that publishers and website owners want to maintain the traffic to their website so that they can continue operating as usual? Or should we all just accept every Google decision, even when those decisions result in more engagement on google.com, but 20-35% decreases in traffic to the original websites?

Also I'm going to need a citation that the vast majority of people want and get value out of AI overviews. Because that is certainly not the case from my experience.

Well besides them being abusive, the other issue is that AI overviews and answer boxes cannibalize traffic to websites, leading to less conversions for the original content producers. This is pretty well established across industries at this point:

https://ahrefs.com/blog/ai-overviews-reduce-clicks/

What has been democratic about how the internet has evolved over the last 2 decades? Because as far as I can see, the internet has undergone a massive centralization into the hands of a few players with practically no regulation. Especially Google, which can make decisions such as adding AI Overviews to search results leading to millions of websites seeing a ~25% drop in organic traffic in the last few months.

Both are an issue. People don't want AI overviews cannibalizing their website traffic. People also don't want AI bots spamming their website with outrageous numbers of requests everyday.

There is nothing stopping other CDN/DNS providers from implementing similar services and tools to what Cloudflare offers. Part of the reason CF has become so popular is because so many of their competitors don't offer nearly the same convenience for routine tasks & protection.

I find it wild that you focus on CF being a monopoly here when they are providing tools that help publishers not have all of their content stolen and repurposed. AI companies have been notorious over the last few years for not respecting any directives and spamming sites with requests to scrape all of their data.

There is also nothing stopping other CDN/DNS providers spinning up a similar marketplace to what CF is looking to do now.

There are about a dozen Google crawlers that can hit your website for different reasons:

https://developers.google.com/search/docs/crawling-indexing/...

Google-Extended is what is associated with AI crawling, but GoogleBot also crawls to produce AI overviews in addition to indexing your website in Google search.

While the number of crawlers and their overlapping responsibilities makes it difficult to know which ones you can safely block, I should also say that pure AI company bots behave 1000x worse than Google crawlers when it comes to not flooding your site with scraping requests.

I come to the exact opposite conclusion. A few large companies scraping and repurposing original content from publishers kills original content on the internet because it takes out the ability to earn a livelihood from creating original content or running your own ecommerce store that is not tied to a mega-company's platform.

Having perplexity or other AI bots go haywire and sending 10s of thousands of requests per minute to your website (despite you having a robots.txt blocking them) is a giant pain in the ass. Not only does your server costs go up, but your analytics and attribution reports start to look messed up because of all the bot traffic.

Again it sounds like the people who are upset by this really want to publish images rather than web pages.

More like people don't want to lose money because a 3rd party stole all of their content, and then repurposed it to show people before they visit their website.

On multiple client sites that have > 1 million unique real visitors per month, we are seeing some days where ~25-30% of requests is from AI crawlers. Thankfully we block almost all of this traffic. But it is a huge pain because it add an addition load to your server and messes up your analytics data for what is a terrible return - traffic from AI sources has a horrendous conversion rate, even worse than social media traffic conversion rates.

Google has made it very difficult to completely block their AI crawling by also using the standard googlebot search crawlers to feed data into their AI overviews and other AI features within Google search. Google says there is a workaround but it also blocks your site from fully indexing in Google search. This also also all covered in the article though.

What does an AI crawler do different from a search engine (indexing) crawler?

Many people don't want the extra bot traffic hitting their site that comes from AI, especially when AI chat & AI overviews in Google provide such a small amount of traffic in return and that traffic pretty much always has horrendous conversion rates (personally seen across multiple industries).

While they have a Github repo, it is incredibly annoying how it's almost impossible to find any documentation around deployment/hosting. Unless I missing an obvious page

I believe Tesla thought they would be further along with FSD than they are

Well Musk has been promising that it will be fully ready next year for the last decade. Which sounds like a horrendous expectation to work under as an engineer.

I don’t believe that the team are lying.

That would be really crazy if the team behind a crypto startup just lied about their business and intentions - certainly has never happened before!

Sounds like the name of a startup thats going to put fence contractors out of business in a few years before getting sued for not classifying their workers as employees.

That's just not true and conspiracy nonsense.