obelix_

No idea what cost that comment refers too. If you are Facebook or YouTube or Twitter or Amazon sure you need your five thousand man PAID army of content reviewers to keep the data clean cause the data is mostly useless. But if you are running Wikipedia search or Stackoverflow's search or the team at the library of congress please go and take a look at the size of these teams and their budgets and their growth rates.

Just temporary. Nature did not need to invent cloud computing to perform massive computation. The speed and data involved in computation going on in a cell or an ants brain show us how far we can still go on desktop. The breakthroughs will come.

In the meantime (unless you are dealing with video) most text and image datasets out there that avg Joe needs can easily be stored/processed entirely locally thanks to cheap terabyte drives/multicore chips these days. People just haven't realized there isn't that much useable textual data OR that local computing doesn't require all the overhead of handling millions of requests a second. This is Google problem not an avg Joe problem that is being solved with cloud compute.

It makes sense if you are handling millions of requests from all over the world per second and need failovers if machines go down.

But...if you just want to run your own personal search engine say...

Then for Wikipedia/Stackoverflow/Quora size datasets (50GB with with 10GB worth of every kind of index(regex/geo/full text etc) ) you can run real time indexing on live updates with all the advanced search options you see under their "advanced search" page one any random Dell or HP Desktop with about 6-8GB of RAM.

Lots of people do this on Wall Street. People don't get what is possible on desktop cause so much of it has moved to the cloud. It will come back to desktop imho.

You have to examine your own assumptions too -

1. You assume stopping "revolution" is the goal. It is not. All you have to do is look at what outcomes "revolutions" in an info saturated/low attention span/consumption culture based society have produced in the last 20 years.

2. In 2013 the Washington post did an estimate on how much was being spent on the surveillance society in the US. Everyone reacted in disbelief. Do you know by how much that budget has increased in the last 5 years? Are these people just brain-dead to be spending this kind of cash? Ofcourse not.

The US has made its share of mistakes over valuing freedom and squandering potential. China is overvaluing control and will make its share of mistakes too. We have to learn from both sets of mistakes to arrive at the right balance of where society should go in high noise info saturation environments. These are very new environment that society hasnt been in before and the right path ahead is not as obvious as people think.

Just cheaper these days to buy them from India. You could fly there, get a top quality doc/audiologist to check you up. The same top of the line aids cost around 2-3K max. Do the touristy stuff and fly back all for less than what it costs to get an aid in the US. "Medical tourism" they call it.

This where I tell couples "it takes a village". In a healthy working "village" there is always someone around who will compensate for weaknesses in both partners that aren't easy to fix. Modern couples have this highly misguided notion that between themselves they can work their way through complex things they aren't trained to handle. The larger their support group (parents, in laws, family, kids, friends) the better the outcomes.

The fix is not complicated. I have no issue with my search data/post viewing history providing me better results or better ads.

The real damaging data to people and society at large is a different set of data. It's the publicly visible counts next to every thought and utterance reinforcing misguided beliefs and behaviour up and down the food chain constantly. Any experienced shrink, psycologist or educator, marketing/PR expert knows applying the right amount of feedback at the right time is critical to how people process info.

Remove/delay/reduce the visibility of like counts/view counts/upvotes/retweet counts that are displayed and the world will be a different place overnight.

I don't know about trust being the root. But the argument can be made that society is very different. And the Fred Rogers model won't have much of an impact given the environment of hyper stimulation/24x7 distraction that consumption culture creates.

Fred Rogers was critical of it even back then. Since then, consumption culture has exploded, objectification of women, violence, escapism whether it is in games, drugs or your social media, news streams is at an all time high. For a Fred Rogers type message to get through the volume dial on all this other stuff has to be dialed down. Otherwise the very valuable message gets lost in the noise.

Interesting data there probably for a social psychologists/sociologist to dig into. More interesting than the amount of cheaters in a population or whether thats a fixed amount is how many go on to win and influence gaming culture (both the player and the designer).

Once upon a time I could find blog posts of people reviewing things, travel, software, equipment etc. Now I just find pages and pages of review aggregators. The advantage of finding a blog was didn't take more than a few clicks to figure out if the blogger and me shared thinking, lifestyles, financial means etc. Now all that is gone. We just have ridiculous aggregation.

Much more than just personal privacy. When CEOs, politicians, judges and generals use the internet too do you really want to be the guy/a company that gives them that call? The incentives are all messed up.

Yup good move. TV news orgs need to do this too. Next too all their flashing banners and scrolls. Especially when the talking heads of both side are raging. I'd really like to know if Coke or Apple or whoever is footing the bill.

humanetech.com/take-control/

Give it some time.

Kim Kardashian has a 110 million followers. After the Russians (or whoever...maybe just that "400 pound dude in the basement") get her to become the next president I am waiting to see what boywonder shows up and says to Congress.

We know what he and his exec staff thought about fake news before the election. Now who here thinks fake imagery is not a thing? Just take a look at the content produced by the top 100 accounts.

There is going to be a special place in history for the damage boywonder has done. People will have to invent new terms to describe it. Nothing like it has happened ever before.

Cause handling people and their issues is about 5000 times harder than handling code. Your code doesn't get cancer, run away with a coworker, go bankrupt, have opinions, lie, steal, needs to be told how to use a restroom, needs protection when it's layoff season etc etc etc

Good points and well put.

The people/environment around the person going through the trauma makes a big difference to outcomes imho. And usually it "takes a village". Because though most people who care will want to help, they each bring different strengths and weaknesses to the table. It's a magic combo of the different strengths that make a diff. And realising that and raising group consiouness about it I agree is badly needed these days.

And that is why psycologists are winning Nobel's for economics these days. Tomorrow they will hopefully play a bigger role in engineering and tech.

Starbucks stores when initially designed, where intended to fill the need for a "third space" outside of home and work. Not everyone wanted to hangout at a bar. And not every city and town had such spaces. Which is why Starbucks worked. People paid for that space not really for the coffee.

If the site is mediawiki based check out kiwix. They have tools to download the whole site with search index. Also tools to search/reopen and render pages from the dump.

I suppose similar tools have been created by the internet archive folk.

But you are right it should be much easier to archive stuff in 2018 and it isn't esp thanks to all the JavaScript and XHR happening.

Edit: I just took a look at kiwix (been a while) they seem to also now archive stackexchange sites not just mediawiki...so looks like they have different archiving tools for different sites.

Thanks for the links. Needs to be reposted often.

Hopefully the day will come where we don't have to become grad level psycologists to handle our new info environment. The professions I was thinking about were air traffic controllers, soldiers, pilots, cops etc whose training involves coping with constant high noise environments. We have lot of lessons to learn from them and their training systems.

I have a feeling psychologists("choice architects"), socialogist etc with software experience will soon start making an impact in dismantling some of the misguided pieces of the current architecture.

Just wrote this 3 days ago - https://news.ycombinator.com/item?id=17325760

Everyone these days is being overloaded with an amount of information -

they will never be trained to handle

that no previous generation has ever handled

What we are watching unfold across any and every issue, are symptoms of people breaking down due to that overload.

To handle this new over saturated info environment (which up until now has been sold to everyone as a good thing) one must look to professions that are trained specifically to handle such overload. And there are many.

Cause now if NASA needs some extra funding they just need to convince Kim Kardashian (or whoever replaces her day after tomorrow on the top of the pyramid) of its importance. It's going to happen. Just watch.

Well that's if you assume this can't be gamed to prop up the non superficial. Bill Gates or Obama constantly used influencers on social media to get interesting messages out to the masses.

What I don't like is that the most hyper, overenthusiastic, over the top characters send a signal to kids that it's the only route up the ladder. It is not.

I totally agree. The information environment we live in is extremely damaging.

Back in 2006-7 I attended a conference where there was a big panel on what the future of news was going to look like. Lots of newspapers were shutting down. They made 3 predictions - news would be replaced with pandering to the lowest common denominator in the arms race for eyeballs, ads and marketing content looking like news(fake news) would exponentially increase, real journalists would be replaced by the talking head/oped class.

They concluded that the only way out, was to hope that revenue generated from the transition from print to digital/mobile would prevent these three things happening.

Its been a decade now and that transition is more or less complete and many have figured out how to stay afloat and yet all three predictions seem to have come true.

The subject you pick and the reaction of the people around you to your initial attempts makes a big difference here.

People underestimate this.

So pick subjects that interest you or are sure to tickle/provoke/excite the people you intend to show the work too.

In my experience getting ppl to admit their issues isn't the way to go. Showing them a path exists to overcome the issues, without a requirement for them to admit anything is. Understanding that is love.

And that's hard work. "Work is love made visible"

We are still in the stone age when it comes to search.

Ask Google who played the mens semifinals of Wimbeldon three years ago and Google will tell you it indexed 6 million pages to provide a link that may or may not have the 4 names I am looking for. Why is it doing all this pointless work? And why is it that dumb in 2018?

We have got so used to what it does that lot of people have stopped asking questions about how it does things and wether all the stuff it does is required.

Wolframalpha, Freebase/SemanticWeb/Wikidata/dbpedia approaches, NLP/NLU are still very underdeveloped and untapped.

Having open and distributed indexes like we see in nature with DNA is also totally unexplored because of Google type centralised index monopolies in various domains. It just takes a Gig or so to store a local offline index off all Wikipedia or Stackoverflow pages. And given the massive RAM and hard disks everyone has these days why aren't we seeing sophisticated local offline search apps?

The internet is getting exponentially more noisy day by day and in many ways its easier to find quality info going through a top notch library's index than wading through Google's. So there are lots of blindspots and areas to explore in search right now imho.

Thanks for the link. It's good to have someone point these things out when one gets carried away :)

Agree but there are two new issues today that I can think of -

1. Targeting is much more sophisticated. Today you can round up every lunatic who hates squirrels or whatever by tomorrow morning and influence what people need to focus on.

2. Messed up social signals i.e. like counts/views/retweets/upvotes etc are just misguiding people left and right. If I don't know what to make of something/too busy/too distracted etc I fall back on these highly inaccurate proxies to make decisions. This is happening all the time and not just to the poor or semi-literate but too highly educated folks. Everyone is unconsiously nudged to support X or Y cause, org or person purely based on these numbers. Take those numbers away and it would be a very different world.